Yes of Course your Course is Rigorous, if you say so

If you write recommendation letters for undergraduate students applying for graduate school, internships, and the like, have you ever included the fact that the student got a good grade in your rigorous, challenging, demanding class?

And if you wrote that, did you back it up with data? Or did you mention the fact that your entire institution is so uber-prestigious that the students are all super-smart, ergo, anyone getting a high grade in your rigorous class is exceptional?

I don't think I have ever described one of my own classes like this. Perhaps my classes are not rigorous. Perhaps I don't think my saying so would be compelling, even if they are. 

So, how do you feel about reading recommendation letters in which someone describes their own course as rigorous (challenging etc.)? Are you convinced by this? If not, what would convince you? Data? That the student was only one of (specified low number) out of (moderate to high number) to get a (high grade) in a course? 

Some (few) institutions give some useful information in transcripts about grade distribution and class size, but most do not. So we are left with the self-described rigorous professors to guide us.

Perhaps I sound a bit cranky about this. I am not really all that cranky about it. Compared to the heartfelt stories of inspiring uncles and chemistry kits that inspired the applicants as children to pursue a passion for science, including graduate study, the self-described rigor -- although surprisingly common -- is more amusing than annoying. 

(And I certainly don't hold it against the applicants, who are, after all, typically have few choices of letter writers and know nothing of the letter-writing skills of their referees.)

Tenure Times

A mid-career reader wants advice about writing a letter as part of the tenure evaluation of a candidate at another institution. In particular:

.. I don't know what the usual length, format, etc. is, and if/how tenure committees "read between lines" for certain cues, i.e. what do I need to do so that I send the intended message without inadvertently compromising it?

I have touched on this topic before, but I don't think I have ever discussed the specific information of length, format etc. My earlier posts on this topic started in 2007, then skipped a few years, then picked up again in 2010 with not one, not two, not three, but four posts on this general topic, then decreased to one in 2011.

Here are my answers to the reader's questions, although of course I hope that others will provide alternative information and suggestions from other disciplines:

Length: One page is too short, but how far you go beyond one page depends on how much substantive information you can provide -- this may or may not be a function of how well you know the candidate and/or the candidate's work. I think it may also be discipline-dependent. In several of the physical science and engineering fields with which I am familiar (and for which I have read external letters as part of the tenure evaluation), 2-3 pages (single spaced, on letterhead, 11-12 point font) is pretty standard, unless someone has really detailed knowledge of a candidate, in which case the letters may be slightly longer (but not by much). In a few other fields, however, very long letters seem to be the norm. For example, I have seen some astoundingly long letters for candidates in the math department. In these letters, the evaluators provided detailed descriptions of every article or other type of work produced by the candidate, in some cases taking us through proofs step by step. Some of these letters have lots of equations and read like lectures. I am not sure that happens in m/any other discipline (?).

Format/content: The request-for-letter cover-letter might provide some clues as to the desired format. Do they want you to address specific questions or topics? If so, you can do this if you want, using the questions as your framework for the letter. Or you can ignore the specific requests and write what you want. One thing that is good to address up-front, even if you are going to be a loose cannon with the rest of the letter/format, is how well and in what capacity you know the candidate. This sets the context for you letter, and is important information for people who will be reading the letter. If you don't have any other specific guides about format and are wondering what to write next, you could pick out a few publications (articles, conference proceedings, or whatever is most relevant) and explain why these are interesting and/or significant.

Something I do look for in the cover letter is whether those requesting the letter want me to comment only on research or also on other things. I may not feel that I have sufficient knowledge of the candidate's teaching and service (the typical 'other things' besides research), so I may not provide an opinion about this, even if asked, but at least I will know what the expectations are. This can be important, for example, if the request is strictly for comments on research/scholarship, in which case you may want to avoid mention of how great this person was on the organizing committee for the Science Conference Workshop Panel Thing. 

I wouldn't worry too much about the reading-between-the-lines issue. Some people do this no matter how you write the letter, and there's no point in getting psyched out about something you can't predict. I have seen letters that I thought were an unambiguous endorsement of a candidate -- letters packed with strong positive statements and substantive examples -- only to have a fellow committee member say But if they really thought X should get tenure, they would have put the word "very" in front of "spectacularly outstanding pioneering genius superstar".

So, don't worry about it. If you want to send a mixed message with both positives and negatives, just be clear about this and about your final opinion (Do you endorse this person for tenure or not?). If you want to be entirely positive, use lots of awesomely positive adjectives. And if you think the candidate does appallingly bad and pointless work, I am sure you can find some equally awesome adjectives to convey that. If you think they are mediocre, say so.

Probably the biggest pitfall -- in terms of sending a message you don't intend -- is if you compare the candidate to so-called peers. You may be asked to do this, or you may want to do this even if not asked. I refuse to do this because I think it is nearly impossible to do it in a fair way, and I have seen a few examples in which a letter-writer wrote X is a spectacularly outstanding pioneering genius superstar just like Z at Other Great University and I therefore support X 100% for tenure at Your University, only to have a committee member say But I think Z is an idiot..

If X is greater than or equal to Z, and Z is an idiot, then X must be.. some answer that depends on who is doing the math. Yes, I realize that this is contradicting my previous suggestion to forget about predicting what letter-readers might do, but I think this particular issue -- that of comparing people -- is a real mine-field.

Is anyone freaking out? Please don't. These are outlier examples that occur and are typically stomped on by the sane faculty members, who, believe it or not, have outnumbered the others in every case with which I have personally been involved in STEM disciplines. That is, I have not seen the outcome veer negative because of irrational read-between-the-liners.

My main advice is: just write a sincere ~2-page letter that has substance to it (examples, incidents) and an unambiguous statement of your opinion at the end and/or beginning of the letter.

Goes Without Saying

The other day, less than hour after I had provided a recommendation for an undergraduate, saying in the recommendation (essentially) that the student was mature and thoughtful, showed good judgment, and was smart, I stood on a busy street corner near campus and watched this particular student ride through the intersection on their bicycle: no helmet, no hands, ear buds in ears, ignoring traffic signals, trusting that the cars, buses, and trucks would somehow not be in the way.

Ah well, this student is young and immortal, that goes without saying. And the things I did say in the letter are still (mostly) true, when considered in the relevant context.

On A Related Topic

The topic of Letters of Reference seems to pop up rather often in the e-mails I get from readers, no matter how much I have already written about this topic. I get questions about writing them, reading them, asking people to write them for you.. The number of ways that this topic is fraught with anxiety and pitfalls is truly impressive.

I seem to have written 29 (now 30) posts on the topic in the past 5+ years. Or, at least, I have labeled 30 posts as such. Previous reference-letter-centric topics have included:

- How (whether) to compare the individual in question with others: peers, people with the same gender, the most brilliant people on the planet in a particular field..

- The most difficult types of letters (for me) to write, and the ones I most enjoy writing;

- The strangest (and most inappropriate things) I have read in letters of reference (I hope I have not written anything that would show up in someone else's list of such things);

- How to assemble a good list of people to ask to write letters of reference for a tenure case;

- How much I hate filling out the forms for grad applications that involve my comparing the applicant with the top 1%, 2.5%, 8%, 25%, 50%.. of students I have ever taught, or that have applied to grad school, or that I can remember, or something, not to mention also having to 'grade' the applicant on a host of Important Skills, from writing to ethics to working well with others. And yet, as much as I hate the ones with 57 categories, I also hate the ones with too few; e.g., I saw a recent one that lumped writing/speaking ability and wanted a single rating for this category.

- Writing letters (as a student) for an advisor, past or present;

- A plea for those who ask professors and others to write (many) letters of reference to send a brief e-mail telling them how things turned out, what you decided etc. Maybe the letter-writer doesn't care, maybe they do, but if someone took the time to help you out, even if you consider it "part of their job", at least have some brief communication with them later.

- What to do if a letter-writer asks you to write a first draft of the letter? (do it, it's just a draft);

- A classification of Letter Writer Types, from most obnoxious to most helpful;

- and let's not forget the Letter of Reference writing contest of December 2009.

That's a lot, and is just a partial list. Is there anything else? In fact, there is. Has anyone else encountered this situation that recently came to my attention?:

You are looking at the application (doesn't matter for what) from someone who worked in industry for a few years. The applicant worked for a small, specialized company in or near their hometown, and has a very positive letter from a top executive in that company. The applicant and the executive share the same last name; it is not an unusual name, but it is also not one of the most common ones. There is no mention in the letter or anywhere in the application that the applicant and the letter writer are related. What would (did) you do?

(1) Assume they are not related and take the reference letter at face value.

(2) Strongly suspect that they are relatives and dismiss the letter as possibly more unobjective than most such letters.

(3) Not care either way; letters are mostly filled with partial truths anyway.

LoR Lore

Today in Scientopia, a discussion of the phenomenon and consequences of Late Letters of Reference.

Imperfectionist

In Scientopia today, a classic topic: mortals writing letters of reference for mortals.

Compared to What?

It is not uncommon for a request for a letter of reference -- for a faculty position or as part of an evaluation for tenure and/or promotion -- to include a specific appeal for a comparison of the candidate with others

Here are some examples of how this request is worded in instructions to external letter-writers:

Standard model/general: How does Dr. X compare with other individuals in his/her field at similar career stages?

Standard model/more specific: How does Dr. X compare in terms of research achievements, standing, and potential with other individuals in his/her field at similar career stages?

Advanced model/domestic: How does Dr. X compare with the best people in the US at similar stages in their careers?

Advanced model/global: How does Dr. X compare with the best people in the world at similar stages in their careers?

Advanced model/comprehensive: How does Dr. X compare with leaders in his/her field with respect to both current and potential future standing in the field, nationally and internationally?

I have worded these as questions, but the same thing may also be worded as a statement: "It would be very helpful to our evaluation if you would compare Dr. X with ..."

A common variant requested of current or former advisors of an applicant for a faculty position is for the advisor to compare the applicant with other advisees.

I don't have any particular compelling reason to argue against this practice, but, when writing letters, I ignore this request.

Many people comply with the request, and that is fine, but as a reader of many letters containing comparisons, I can say that I rarely find the comparisons useful, even if I know all the people involved (applicant, letter-writer, noted peers).

In my previous experiences on committees evaluating candidates for tenure, promotion, or awards in disciplines other than my own, the names of compared-peers are typically meaningless to me, and are used only for the departmental evaluation stage. I can maybe get something out of the peer's institution, but even here, I'd have to know the field fairly well because there are universities that have top-ranked departments in some fields but low-ranked departments in others. This applies to all types of universities, of varying levels of prestige and ranking overall.

One thing I find fascinating when I read these letters is that some candidates/applicants are compared to a completely different set of peers in each letter. For tenure/promotion, there might be 6 or 10 or more letters containing these comparisons for one candidate, each one listing different people as peers. That means that there is little agreement about what constitutes a peer and what constitutes an individual's "field".

This is particularly true in science and engineering fields that are highly interdisciplinary and that may involve very large numbers of researchers. Letter writers may be selected for their expertise in different aspects of an applicant/candidate's research interests, and so the letter writers may be familiar with different sets of peers.

Some letter-writers who make comparisons are quite general in their response: "Dr. X's research accomplishments are comparable to W (University of A), Y (University of B), and Z (C University)."

Others are more detailed: "Dr. X's research accomplishments are greater than those of W at the University of A, but s/he has not been as productive as Y at the University of B. Nevertheless, I rate his/her creativity as superior to that of Z at C University, although all of these are second-tier compared to T at the University of R." It is rarely clear what the basis of such detailed comparisons is.

Except when it is very clear, such as when the letter-writer specifies that s/he used a specific metric, such as number of citations, papers, or the h-index.

I am usually quite skeptical about claims that someone is "the next [insert name of genius-successful-famous person]". If the claim is based on the types of problems or methods used, or is at least backed up by some achievements to date, OK.. maybe. If, however, the letter is a vague prose-poem to someone's awesomeness, these comparisons are as believable as book blurbs.

Dr. X's last article in Nature could have been written by a young Charles Dickens!

Dr. X really knows how to write a proposal -- like Marcel Proust if restricted to a 15-page limit!

Dr. X's CV is an electrifying page-turner! The 2008 article in The Journal of Rigorous Science could have been written by Stieg Larsson!

Dr. X is that rare anomaly: an evocative and inventive scientist and prose magician, like Nikola Tesla and Virginia Woolf rolled into one!

The Women's League

Today's post is at Scientopia, where I discuss the following statement, from a real letter of reference sent for a recent candidate for a faculty position:

[the applicant] is in the same league as other top female graduates [from this department]

Spelling it Out

As I have been writing letters as part of the tenure and promotion evaluation of assistant professors at other institutions, I have been struck by the great variation in the "tenure code" documents I have been sent to help me write my letters. Typically, these documents outline an institution's or a department's criteria for tenure and promotion, but the level of specificity of these documents varies a lot.

At the vague end of the spectrum are documents that state that the candidate must show evidence for "scholarly achievement" (or words to that effect). This is useless for my purposes, and I just write my evaluation however I think best. In my experience, these documents are the most common type.

At the definitive end of the spectrum are documents that state that the candidate must have x papers every n years in "respected journals" and at least y grants from external funding agencies. In these cases, which I rarely encounter, I don't really understand the point of an external letter. If the candidate met the criteria and is going to get tenure, why ask for anyone's opinion? Does it matter if I think the x papers were flawed and uninteresting? If not, then don't waste my time. If it matters, then rewrite the document to include some vague statement about "scholarly achievement".

The definitive tenure criteria likely result in fewer tenure appeals than the vague tenure "criteria", and the vague ones surely result in higher levels of anxiety and uncertainty for tenure-track faculty, but the definitive ones have problems as well. For example, unless there is a way to specify the quality of the "respected journals" (probably by impact factor, for lack of a better measure) and the contribution of the candidate to a multi-author paper and whether it is good or bad to have lots of co-authors (including students), there is little point in specifying the number of papers that must be published in n years unless that really is all that matters.

In that case, these documents might as well just say: To obtain tenure, a candidate must have their name somewhere in the author list of a paper published in some journal that someone respects.

I am actually ambivalent about these documents, speaking from the point of view of a letter writer. I know they must exist, but there really is no good way to construct them to be useful to the letter writer (or the candidate): too vague is useless, and too specific is strange and leads to more questions (most of which can't be answered).

As a candidate, you have to find out the 'unwritten' information by figuring out the norms of your department and institution. This is typically accomplished by having an effective mentor, by talking to other assistant professors, and by having a candid talk with your department chair and mentor at various stages along the road to the tenure evaluation. You can also attend informational talks given by deans or deanlets, although, as an assistant professor, I found such workshops disturbing because the information in them conflicted with what my department chair had told me about some issues (e.g., the selection of letter writers). Even so, I identified these conflicts of information, talked to the chair about them, he talked to the dean, and all was eventually sorted out.

As a letter writer, I don't know the 'unwritten' information, and I'd rather not guess. All I can do is try to write a fair letter based on what is in the record and on my perception of the quality of the candidate's research. The people at the candidate's institution will have to sort the rest out for themselves, depending on what they think my criteria/standards for tenure are likely to be; i.e., they can discount my letter (if my criteria seem too harsh), take it seriously (if they think my opinion is well supported), or pick out the parts that agree with their own opinion.

Mostly, I just need to finish these letters so I can stop obsessing over them (one in particular is much more difficult to write than I expected), and then I can move on to my next adventure in Professional Service Activities.

Difficult Cases

Requests for letters of evaluation for tenure candidates started arriving during the summer. The process typically starts in the late spring or summer, when the dossier is assembled and the letters are requested, with letter due dates in the early fall so that departments can vote and pass their recommendations along up the administrative chain.

I almost always say yes to requests to write letters for tenure cases. In fact, I think I have only said no once, for someone who was too far outside my field and who had (in my opinion) been quite unproductive. I thought it best if people who were more expert in that field evaluated whether the quality of the papers more than made up for the low quantity of papers; I did not feel confident that I could do this.

Also, I was given a very short amount of time to write the letter, and could not have done a thorough job. I think it is important for these letters to contain something new and informative that cannot be gleaned from reading the CV, so writing these letters can take time.

Declining to write a letter for someone's tenure evaluation can be seen as a blot on their record unless you have a really really good reason for declining. Department and university P&T committees may pore over decline-to-provide-a-letter explanations, trying to determine if the declination reflects a negative opinion. It may not, but some suspicious faculty may still wonder: OK, so this person is really busy preparing for their Senate confirmation hearing, but if they really thought the candidate was great, wouldn't they take the time to write the letter anyway?

So I try to write the letter if it seems at all reasonable to do so in terms of my expertise and/or time.

Although most letters are easy to write in terms of content, some letters involve a bit of agonizing over wording and tone. For me, the most difficult letters to write are:

1. letters that are not entirely positive. In these cases, there is always a question of whether to accept or decline writing the letter in the first place. I think it is important to be certain that a negative letter is also a fair letter. If my opinion is not positive, is there a good reason for this? Do I have the necessary expertise to evaluate an individual's record? Can I be objective, or am I unduly influenced by an irrelevant dislike of the individual or his/her research topics?

2. letters for institutions that are not peer institutions. It is common to be asked to comment on whether the candidate would receive tenure at our own institution. If the candidate is at a peer institution, this is not typically a difficult evaluation to make. For candidates at institutions that have less emphasis on research and/or that have less awesome facilities and/or a smaller graduate program (and rare or no postdocs), this evaluation is more difficult. I've written before about how it can be challenging to state that the individual in question would probably not receive tenure at my institution, but is a good candidate for tenure at Lesser University. By definition, that's a rather patronizing and obnoxious statement. I have seen P&T Committees agonize over such statements, even if the rest of the letter is positive and even if the letter writer is at a university that is famous for eating its young (faculty). These statements are not to be made lightly or without justification.

3. letters for candidates up for early tenure for reasons that are not obvious. Faculty up for early tenure are evaluated by the same criteria and standards as those up for tenure at the normal time; you can't give them a break for not having as much time, nor can you hold them to a higher standard. Some assistant professor superstars get early tenure because they have such outstanding records; it's easy to write glowing letters for them. But what do you do if you get a dossier for someone who looks like they are on track but could really use another year or two to demonstrate that better? What if you look at someone's CV etc. and you have no idea why they are up for early tenure? Unless you are totally off-base with your criteria and standards, perhaps there are some departmental politics at work and you don't know what these are. These are difficult letters to write because you might have been able to write a really positive letter in another year or two, but as it is, you can realistically only write a somewhat vague letter about how this person shows some promise. In these situations, I just write what I think anyway, and let the department sort out its own political situation however it wants.

The letters from senior faculty at other institutions are just one element of a tenure dossier. I think it is worth putting effort into writing the letters, I don't believe that any one letter will either sink or save a candidate. It's just another data point, so I might as well make it a useful and interesting data point, without either over- or under-estimating the value of my opinion.

Summer Letters

You would think that summer would be a time when I did not have to pore over anyone's CV or read letters of reference and such, but no; not this summer (or last summer or, apparently, next summer).

So, I read 58 letters of reference this summer. For the most part, these were the usual "X is great, blah blah blah" letters, so when there was something a bit different, that something really stood out. For example, after reading one particular letter, I wondered

WHO WRITES THIS KIND OF THING IN A PROFESSIONAL LETTER OF REFERENCE THAT IS SUPPOSED TO BE ABOUT THE RESEARCH ACCOMPLISHMENTS OF A PROFESSOR OF SCIENCE?:

Bob and his wife have spent many vacations over the years in Bavaria, organizing their walking routes to coincide with the locations of breweries.

Apparently someone does write this kind of thing; I saw it with my own eyes.

Why would I care that Bob likes beery Bavarian rambles, or even that he has a wife? Was the letter writer grasping for things to mention about Bob? Was the letter writer drunk on Bavarian beer when writing the letter? Are we supposed to think that Bob is a cool guy? Oh how we wish we could go hiking in Bavaria with a wife and drink beer, just like Bob?

The Bob-Bavaria-beer statement was completely irrelevant to the purpose of the reference letter. The letters were for moderately senior to very senior professors being nominated for an award related only to research accomplishments. The nominees need not have any hobbies or interests outside of research, and need not have any particular interest in Bavaria or beer. We, the committee reading the letters, need to be convinced that the nominee's research accomplishments are enduring, extensive, and awesome; that's all.

Who Cares What They Think

Some of my readers are in the process of selecting Names to put on lists of possible letter writers to comment on their tenure dossiers. Some of my readers are obsessing about this, as I myself once did.

Some people who evaluate these dossiers think that such letters are essential to the process; others think the letters are essentially worthless. Yet it's hard to imagine any application or nomination process, including tenure and promotion, without them.

So: How to choose the Names that go on the list? Previous discussions of this topic have covered such issues as: Should you leave out obvious names in the hopes that those will be the ones selected by the department/tenure committee? (Short answer: Yes, maybe you should do that to some extent, but don't leave out all the most obvious names).

Today's discussion, by request: Is the degree of "famousness" of the letter writer important? Do such letters have more weight than letters from less famous people, or would a very positive, thorough, and convincing letter from a qualified but less famous person have the same (or more positive) effect?

Some people do weight Letters from the Famous more and some do not.

I have witnessed many times the phenomenon in which someone on a committee is awed that an Academic Superstar -- a Nobel Laureate, for example -- wrote a letter for an applicant or nominee. In extreme cases, letters by former students of Nobel Laureates are given particular weight, as if the genius* of an adviser is automatically passed on to all students.

[* Let's not discuss whether all Nobel Laureates are geniuses.]

This happens, but it is always controversial. Someone else on the committee typically rolls their eyes and makes a sarcastic comment. On all-university committees (e.g., for awards), humanities faculty are particularly good at rolling their eyes when another committee member slavers over a letter from a Nobel Laureate in Science, although I wonder what they would do if someone had a glowing letter from J.M. Coetzee or Toni Morrison.

On the various committees on which I have served for hiring, promoting, and awarding, many of us are impressed by letters from Big Names if -- and this is a big if -- the letter is thorough and shows that the Big Name really is knowledgeable about the candidate's research. Certainly such a letter is useful if the BN has interacted with the candidate and has substantive things to say.

I am, however, profoundly unimpressed by letters from Big Names if the letter consists of little more than a terse statement to the effect of: "To whom it may concern: By taking 2 seconds out of my day to instruct my assistant to affix my electronic signature to this letter, which says nothing but which has a really nice logo, not to mention my illustrious name on it, I herewith bestow my acknowledgment that the applicant seems to exist and probably does pretty good work because he/she is associated with my lab/institution/colleague. Please find attached my impressive CV."

Such letters are of no use. If you get a letter from a Big Name, you should hope that the BN will take the time to write a letter with content, otherwise you may be better off with less famous but conscientious letter writers.

But of course you don't know if the less famous people write thorough letters or cursory letters either.

How do you know if a letter writer is conscientious so that you can plug the relevant variables into an equation and determine who should be on your list? I know I have discussed this somewhere before.. somewhere in a post deep in the archives .. but I am pretty sure my answer has not changed: in short, you (probably) don't.

If you are trying to decide which people should go on your list of possible letters writers, whether or not you are agonizing between listing Professor Nobel or Professor Non Nobel, maybe you can get some information from others who recently went through a similar process in your field, maybe you have a mentor who has seen letters from these people before, and/or maybe you have had varying degrees of interaction with these people and can guess how much effort they might put into a letter.

If you really don't know and can't find out through reasonable means, you might as well flip a coin, or list Professor Nobel even though you don't know him/her well because it would be cool if that worked out, and then don't worry about it (too much) because your dossier will be seen by faculty who have seen thousands of letters of reference and have experience sifting through the rubble and won't hold one uninformative, terse letter against you.

My advice, in summary:

- A detailed and impressive letter from a respected but not cosmically famous person is better than a cursory letter from a Cosmically Famous Person. Committees are influenced by the prestige of titles, institutions, and awards, but if you have awesome letters from respected people in and beyond your field, don't worry that you haven't yet managed to hang out at the conference hotel bar with Professor Nobel.

- Make sure that as many names on your list as possible are people who are likely to be able to write knowledgeably about your research. If you think it is important that you have at least one Big Name, go ahead and put one on your list if it is at all reasonable for you to do so based on your research field and accomplishments. But: be conservative about the number of BNs on your list, unless you know them well.

Retract?

An FSP reader has a conundrum.

Imagine that you find yourself in this situation (anecdote condensed and edited from the original e-mail): A friend and co-worker has lost his/her job. You think this person has been treated unfairly, but you don't know the official facts of the case. Rumor has it that this person had done unethical things, but you have worked closely with this person and had found him/her to be very professional and reliable. On the basis of your own very positive professional interactions with this person, you write positive letters of reference for faculty jobs.

You later receive credible information indicating that this person was in fact lying about a great many things to a great many people, and that there was substantial unethical conduct. You can no longer stand by the statements in the letters of reference that you sent on this person's behalf.

This person claims to have found a position that will begin in the fall, but you don't know if this is true and there is no one at that school whom you can ask informally. A web search doesn't turn up anything, but it probably wouldn't for a recent hire anyway.

Do you have an ethical obligation to contact the schools to which you sent the positive recommendations and inform them that you can no longer stand by the statements to which you signed your name? Would search committees and department chairs want to know this?

Aside from the ethical considerations, what might be the consequences of these now-discredited letters for your professional reputation, especially if you are an early career faculty member trying to get established? Are the consequences worse if you retract your statement or if you do nothing?

What would you do?

First, I want to reassure my correspondent that he/she should not regret writing the positive letters. The letters were written in good faith, based on personal experiences working with the colleague in question. The letter no doubt mentioned these positive interactions, perhaps providing specific examples. If there were no indications of unethical behavior in these interactions, the letter was a fair statement of what the letter writer knew to be true.

If the letter stayed close to the specific interactions that the letter writer had with this colleague, perhaps a retraction is not necessary. If, however, the letter writer made some broader statements that really should be retracted, perhaps a brief letter should be sent to the relevant search committee chairs and/or department chairs.

The letter need not elaborate on the situation, but could just say something like "Owing to facts that have come to my attention since I wrote the letter on behalf of Colleague X, I can no longer stand by the positive letter that I wrote on his/her behalf. Please retract my letter, if this is possible and relevant to your search." You can decide whether you are comfortable mentioning that you are willing to provide further explanation if necessary, and then list your phone number.

If Colleague X has a new job, there must have been more than one positive letter.

Another possibility is to discuss this with a trusted senior colleague -- e.g., a mentor or the department chair. Perhaps they can intercede in the situation, especially if there is a senior person with more direct knowledge of the unethical doings of Colleague X who would be willing to help figure out the best course of action.

But perhaps various readers will have various other suggestions..

Wasted Time?

Some applications for open faculty positions require letters of recommendation up-front; some request letters only after there has been a first stage of selection based on other information supplied by the applicant. Although I don't mind the first method, I prefer the second because it seems like a better use of time for those writing the letters and for those reading the applications.

I know that things are done in a different way at many non-US institutions, but lately I have encountered variations on this system within the US. In the past few months:

- I received a request for a letter of reference for someone who had already been invited to interview at a US institution. In fact, no letters of reference were solicited until this stage, so my letter wasn't supplementary or an attempt to fill a reference letter quota. The letters were clearly not a part of the interview selection decision. It is unclear what role, if any, they have in the search.

and

- I received a request for a letter of reference after someone had a phone interview but before they were invited for the actual interview. I am glad I am not on that hiring committee, as it seems to involve several extra steps.

I didn't mind either of those requests, although the first case made me wonder why the institution even bothered to get letters if they are so unimportant in the process. Perhaps it is an administrative requirement? Having read thousands of these letters, I must say I can't really blame anyone for not valuing reference letters very much.

A letter request that did annoy me, however, was one accompanied by the information that my letter would not even be read unless the candidate advanced to the final interview stage. In this case, the position is not an academic one, so perhaps I just lack familiarity with how things are done outside academia.

From what I could tell from the instructions, the organization will not read the letters before the final candidates are selected from a huge pool of applicants but wants to have the letters on hand to read as soon as the final interviewees are selected, and hence the request for possibly superfluous letters. I can understand the wish for efficiency, especially given the large number of applicants, but it is a bit of a strange request: Please write this letter even though we are unlikely to read it. We need to be efficient but we are willing to (possibly) waste your time.

Wouldn't such a request lend itself to getting cursory letters or letters that are not focused specifically on the position for which the applicant has applied? I typically take some time to customize each letter depending on what the position is, and in some cases this takes quite a bit of time. Why would I do that if I don't know if my letter will even be read?

I guess I will do it for the same reason we write any reference letters, not knowing if the subject of the letters will get the position.

Despite knowing in this particular case that my letter may be transferred, unread, directly to an electronic trash heap, I think the best strategy is to try to forget about that and write the letter in the hopes that it will eventually be read, the student will get the job, and it will all be worthwhile in the end.

Backfire

You might think that I have exhausted the topic of Letters of Recommendation, and perhaps I actually have and don't know it, but something else that caught my eye in the LoRs involves the attempt by a LoR writer to praise a candidate by stating that the candidate is smarter than the person writing the letter.

Perhaps some will find this refreshing. Professors being modest! Professors admitting that some students are smarter than they are!

But there's a potential problem with this approach. It's not that we don't believe the letter writer. The problem is that we do, and this might undermine the writer's attempt to praise the applicant.

The context is important, of course, in how the letter is evaluated at the other end. Is the not-as-smart-as-the-student professor someone we all know and respect as a scholar? In that case, we are impressed.

Or do we not know this person and therefore don't know whether to be impressed or, more likely, not?

On reading such a letter from an unknown professor, some have remarked "What kind of education could the not-as-smart professor have given the smarter student?". I am not one who worries about this. There are lots of very smart students, and I think it entirely possible for not-as-smart professors to provide a good education to smarter students because, presumably, even if a certain professor has a lower IQ or GRE scores or however you want to measure "smart", that professor has accumulated more knowledge about a particular topic than the student and may even have developed considerable skill at conveying that knowledge.

This reminds me of a conversation I had, early in my professor career, with a first year MS student who was contemplating working with me. After explaining something to him, he said to me "You seem to think you know more than I do*, but we're the same age, so how can that be?" Well.. I guess if you ignore my years as a PhD student, a postdoc, and a professor, maybe you could say that we should have theoretically acquired exactly the same amount of knowledge during our equal number of years of existence on this planet. In fact, what I secretly thought was not even then (in this particular case).

Anyway, I don't personally mind the "this student is smarter than I am" approach to LoR writing, but some of my colleagues have a different view of such statements in LoRs. So, in general, I advise resisting the urge to include such statements in a LoR unless you are widely known for being brilliant, or at least very smart.


* Actually, what he really said was "You seem to know more than me", which was also a true statement.

Who Are You?

Some colleagues and I recently discussed whether you should agree to write a tenure/promotion letter for someone whose work you don't know particularly well, if at all.

One colleague argued that if you haven't heard of someone and you have never read a paper by them before, you shouldn't write a letter because you will have to say that you don't know the candidate and this will likely be interpreted to mean that they are not visible in their professional community. Also, if you haven't heard of someone in a fairly specialized field, can their research really be any good? And then there's all the work of reading the papers you haven't read before. This colleague refuses to write letters for people whose work is completely unknown to him.

Another argued that if you don't know someone, then you can be objective and your letter might have more weight than that written by the candidate's best friends and collaborators who can write detailed letters from the point of view of people closely involved with the research.

Another agreed, saying that he had recently written a letter for someone whose work he didn't know, and in the process of reading papers by the candidate, realized he should know about this person, whose research he found to be interesting and excellent. Being unaware of someone's work can also be a reflection of the letter writer's level of awareness of the field.

It is important to note that this discussion took place in the context of professors at a research university that values high visibility, international reputation, publication in high impact journals etc. etc. We are expected to be known in our field; hence, our discussion, which may not be relevant to other types of institutions.

The discussion took place among colleagues from a wide variety of STEM fields. I suspect that some of my colleagues' opinions relate in part to how likely it is that you have encountered most others who are working in your specific field of interest, at the very least reading a paper or seeing a conference presentation by the people active in your field. In some fields this is expected, in others perhaps it is less common.

I think it is also important to note that I am not talking about whether a letter writer knows someone personally or not. I am talking about whether you know of someone's work, e.g. through publications, proposal reviews, or conference presentation.

I don't have a strong opinion about the issue based on personal experience because so far I haven't been asked to write a letter for anyone whose work I didn't know at all, though in a few cases I had to delve into the literature quite a bit so that I could write a substantive letter.

There was one case in which I was asked to write a letter for a promotion to professor, and I said no, I didn't have time. In fact, I wasn't familiar with anything this person had written in the past 6+ years. I assumed the candidate had been publishing in another field and I did not have time at that point to do a good job reading an unfamiliar body of literature. In that case, I was only given a few weeks to write the letter, which is not enough time in general and was impossible in that case owing to the coincidence in time with a grant proposal deadline and some travel, so I did not feel too bad about saying no. It turns out that this person hadn't been publishing much, so I am very glad that I said no, especially if I had to answer that pesky question of "Would the candidate be promoted at your institution?" (which is only a fair question if the institutions are truly comparable in academic environment , resources, and expectations).

In general, I think that if I were asked to write a letter for someone whose work I didn't know, I would spend a bit of time poking around to see if my lack of awareness was my own fault or reflected a true lack of visibility by the candidate. In the latter case, I would then have to see if I had time to do justice to a letter by delving into publications etc., and then I would decide whether to accept or decline. I guess you can say that, unlike some of my colleagues, I don't have a philosophy of definitely agreeing or declining based only on the fact of not being aware of the candidate's work.

So, if you are at a major research university and are asked to write a tenure/promotion letter for someone whose work you don't know -- perhaps even someone you have never heard of -- would you write the letter or not?

LoR: What I Think

Writing letters of reference (or recommendation) is part of the job of being a professor. We write them for undergraduates, graduate students, postdocs, professors, and other academic people applying for various things (school, jobs, promotion, awards).

It can be a major point of stress for all concerned. The subjects of the letters wonder what we will write. The writers of the letters wonder (in some cases) what to write. Some applicants worry that some letter writers won't submit letters of recommendation on time (if ever); some applicants are justified in this worry. Some professors get annoyed when students ask them to write letters of reference on very short notice and/or provide disorganized, incomplete information.

And has any professor, during the year in which they were being considered for tenure and/or promotion, wandered around a conference feeling paranoid about who was writing their external letters and wondering what they were writing?

And so on. But write them (and read them) we must, in great profusion.

If I had to rank my preference in writing letters of reference, from don't-mind-writing-them (= best case) to don't-like-writing-them (= worst case), my list would look something like this:

top = best (in this relative scale); also: list assumes that I have agreed to write a letter so it does not include the worst case scenarios in which I would not/could not write a letter

• grads/postdocs applying for jobs such as faculty positions

• undergrads who are applying for something (grad school, internship, job) and who worked closely with me on a research project

• tenure and promotion letters for people who are great and whose work I know well

• colleagues being nominated for awards they deserve

• undergrads who are applying to something and who only took 1 class from me and about whom I know almost nothing except what their grade and maybe where they say in class but I feel obligated to write a letter in cases in which the student has no better options for letter-writers
  

• tenure and promotion letters for people whose work I don't know very well* (this is at the bottom only because of the vast amount of time it involves; if we factor out time, the previous one in the list is my least favorite).

But let's not focus on the negative. Why do I "like" (relative term) writing letters for students (whom I know) or postdocs applying for things? I like it because in most cases writing such letters is a very positive thing to do. If you have worked closely with someone for a year (some undergrads) or more (grads, postdocs, some undergrads), you probably have a lot of things to say, and, if you have agreed to write a letter, presumably you have some positive things to say.

Despite the time commitment, it can be a very positive experience for the letter writer to think back on someone's research/education experience, pick out the essential points and examples, and write a well-crafted letter geared towards the specific job/institution to which the candidate is applying. In this case, writing the letter ends up being an affirmative experience, as long as you don't think about the cynical committee members reading 100s of these awesomely positive letters and as long as you don't have to write many many of these letters at any one time, in which case the personal hand-crafted letter thing falls by the wayside.


* Some colleagues and I recently had an argument about this topic: Should you agree to write a tenure/promotion letter for someone whose work you don't know well? A post will follow at some point with elaborations.

More LoR

Thanks to all who sent submissions to the Letter of Reference contest. Here are some more interesting examples, some of them clearly intentionally obnoxious and offensive:

*****************

Dear Admissions Committee:

I am heartily pleased to recommend to your graduate program Ms. Rebecca Hartley.

Ms. Hartley first came to my attention during my course, "The Music of the Spheres," during which time she tried to emulate some of the music we were analyzing, using her iPhone for texting. In 12-tone. I thought at the time that was an indication of a high degree of independence and creativity. Alas, although her enthusiasm was unbounded, her earned grade, which you can see in her transcript, was a C+.

I next interacted with Ms. Hartley when she asked to join my research group. Always willing to work with new people, I happily agreed. Ms. Hartley soon proved herself adept at such tasks as keeping the laboratory's Facebook page (which she instituted) up to date (however, not the lab's webpage). Whenever a volunteer was needed, Ms. Hartley was often ready to do the task, particularly if it involved a trip to Starbuck's. Ms. Hartley's research has made great strides since she's learned to use the laboratory instruments properly. Indeed, she is working on a poster to present her results at the next Sphere Musicology Workshop in 2010. I can't help but feel that Ms. Hartley's manuscript would be ready for submission by now had she only decided not to join her family on Lake Como for the 5 weeks of winter break (after having gone home for 2 weeks of Thanksgiving break), but then when one's mother calls, what can one do?

As for the rest of her undergraduate achievements, I know they are spelled out in her application and listed on her transcript: president of Delta Delta Phi sorority, head of the cheerleading squad (for football, basketball, and fencing), president of the Resident Life Food Committee (coffee division), food writer for the school newspaper (coffee bar reviews are her specialty), president of the iPhone Apps Club, and person-in-charge of the undergraduates' collective Facebook page, "Dickeyville College Experiences." Last year she had the lead role in the annual show, a salute to Dickeyville. It was quite the memorable performance.

In summary, I enthusiastically endorse Ms. Hartley for admission to your graduate program. She will make an outstanding student, in more ways than one. Yours is the program she is most interested in; she is not applying anywhere else. If she is denied admission, she has asked to stay on here, with me, for her Master's degree.

Professor Wimberly Wade
Dickeyville College

***********

Dear Acceptance People,

I am writing to you to extol the virtues of one Mr. Joe Stone. Jim has been an undergraduate researcher in my laboratory since 1996 (?). Since graduating near to the top of his class in 2006, he has worked in my laboratory intermittently (during?) that time.

Jack is a hardworking and very bright individual, who has made numerous positive contributions to our team. As well as being extremely studious, she is a social individual and always manages to lighten the intense atmosphere that permeates a prestigious laboratory such as my own. A butterfly, one might say. This is a refreshing aspect to his personality. One flaw that could be a potential problem for John (especially in a world‐class laboratory with dangerous chemicals at his disposal) is his difficulty in resolving internal conflicts. His sociopathic tendencies could potentially negate his future interactions with colleagues, but since he has overcome his pornography addiction – I believe that this should not be a major issue. However, I feel you should ask him this directly. Joe is a fantastic brain. However, he does need to find the testicular fortitude within himself to succeed. His luck with the inferior sex has also improved exponentially, as has his general taste in aesthetics. I have noticed this personally. He is like a hybrid and good with the animals, however he should be monitored.

His particular research interests have meandered considerably throughout the time I have mentored him. However, I find a lack of ideological consistency within his train of thought. I have placed much responsibility on Joe. He has not broken yet. I admire a man with convictions, especially a man with a thetan level such as Joe’s. He has worked hard to improve this (by successfully calibrating an old e‐meter) and invests a significant amount of time in his faith. Indeed, he is president (?) of our university’s dance troop (? further clarification needed). He has a penchant for cycling and is quite good with the PCR machine.

I am convinced that with Joe’s natural vigour and eternal robustness, she will reach the pinnacle of whatever goal she decides to make. I will support this wholeheartedly. She is the creme de la mer of our crop. In conclusion, x is an ideal candidate for your programme for the reasons I have outlined above. I believe x will be an asset to your department and will thrive at an institution such as yours. I hope you will see x in the same light as I have, by granting him/her a position on your course.

With kind regards and kindred spirits,

CHARLES
Professor Sir H. Charles A. Ruddimutterigenered IX
BA MA PhD DSc MB BS ChB FRS FMedSci FRSL KD CBiol PhD (h’c) (Oxon)
Established Chair and Distinguished Professor of Freudian Zoology
University of Michael Jackson, Neverland, USA
Board Member, BAAADAAAAAAAAASS, Nobel Institute
hcaruddimutterigenerediiv@michaeljackson.edu
‐‐‐END DICTAPHONE TRANSCRIPT.

******

To the Admissions Committee:

I am so excited to write this letter supporting Tommy Terrific’s application to your most prestigious graduate program. Although I am not an expert in your field of Science, I am an accomplished and world-renowned Harry Potter scholar and know a thing or two about magic. My work has earned me an endowed Professorship at a university even more prestigious than yours. Importantly, I also know a thing or two about Tommy, whom I have known since he was 2 years old. From this time, it was clear to all of us on the block that Tommy was not only the happiest toddler on the block, but also the smartest. His true gift is an unparalleled SCIENTIFIC INSIGHT. This gift exceeds even that of his parents, both of whom are world-renowned physicians at my university.

In closing, I urge you to admit Tommy to your graduate program in Science. Based on all that I know of him and his family, he has no choice but to solve the most important problem in Science during his research training. This will bring fame and fortune not only to Tommy, his friends and family, but also to your university.

Sincerely,
The Scholar Next Door

*******

REFERENCE LETTER FOR STEVEN P MORRISSEY

Morrissey is applying to your graduate program and has asked me to write a letter of recommendation. I am quite happy to do so.

My interaction with Morrissey is limited to one course during the Spring term last year, Intermediate Hot Science, which he took as part of the requirements for his Science major, and which I taught. Although I cannot say I know him well, I can say with confidence that, unlike others who are just a hatful of hollow, he is an extraordinarily serious person.

I will be frank and say that my course is a difficult, quantitative, and rigorous class. Nevertheless, although not the top student in this or, likely, any class, Morrissey never seemed to lose the thread of my lectures, which are all very rigorous, despite his propensity to stare blankly at his shoes during class. Furthermore, he persevered in the course despite enduring an unusual personal tragedy; his girlfriend was in a coma for much of the term.

Based on my limited interaction with Morrissey, I would say that his main shortcoming, which might be overcome in a highly structured graduate program, is that he tends to be rather pessimistic. For example, when he asked me if I remembered him from the course last year and if so, would I be willing to write this letter, he said (and I quote): “I don't mind if you forget me. Having learned my lesson, I never left an impression on anyone.” In Morrissey’s case, this apparent pessimism might actually be an unusual form of modesty, as he has also been known to state “I’ll never be anybody’s hero”. I believe he also sings well.

In writing this letter, I find, much to my surprise, that Morrissey has become a central part of my mind’s landscape, at least for the moment. I take that as a positive sign of the strong impression that this young man makes on people who encounter him. Whether that is an indicator of success in graduate school is anyone’s guess at this point, but I strongly encourage you to admit him to your graduate program anyway. Even if you come to regret the enormous investment in time and money involved in trying to foster the graduate career of a dysfunctional person, I have no doubt that you will find the experience an interesting one.

I apologize for the possible lateness of this letter relative to your deadline. When Morrissey gave me the information for submitting the letter, he was vague about when it is due, saying only "How soon is now?", which I prefer to interpret as a profound rhetorical question rather than sarcasm.

Sincerely,

Theo Smiths
Bob R. and Sheila C. McTruffle Professor of Rigorous Science


*******

To Whom It May Concern:

This letter is a strong endorsement of Student X, who is applying to your graduate program. Student X has enormous potential for creative graduate study, and in fact has already amply demonstrated many important skills required for success in graduate research in Science.

My first indication that Student X was special came when I was supervising him in a research project to test the hypothesis that cats are more likely to sleep on top of a warm radiator than on a tray of ice cubes. Student X not only did the assigned tasks in a competent and cheerful way, but also moved the research in new and exciting directions by proposing that we include a variety of felines in our study; in our original plan, we proposed to confine our research subjects to orange tabbies.

Student X continued to exceed my expectations in every way. On numerous occasions I expected Student X to quit, whine, stare at me with an incredulous and/or sullen expression, or start writing a hostile post to his blog when I proposed what many would consider an unreasonable task or deadline, but his response instead was to get right to work, in some cases whistling a tune that surely was of his own invention. And if that wasn’t proof enough that Student X has got what it takes to do mindless tasks for years on end without complaining, consider this: Student X does not have an iPhone and does not even want one.

Student X presented the results of her research at the recent Thermofelinics conference, is intensely focused and motivated, and has outstanding communication skills (including writing!). In sum, Student X is the kind of person you want in your graduate program.

Morris Boltzmann
Professor

(note from FSP: I tried to slip this one past the FSP Editorial Board, without success, so I might as well confess that I wrote it. And the Morrissey one. But that's all, just those two.)

LorR Results, continued

The following are apparently real Letters of Reference, but I think they are worth including here for their entertainment and/or horror value:

REAL LoR 1 (from "seat-of-the-pants scientist"):

XX is a student in my lab. She has made progress this year.

REAL LoR 2:

APPLICANT may be mediocre but her personal problems make it impossible to tell.

MOSTLY REAL LoR 3:

Marc X is a student whose self-perception of achievement is inversely proportional to his actual skill level. He is incapable of teamwork and unable to understand criticisms of his own work. I have had to put up with him in three classes. He likes to ask questions that he thinks demonstrates his superior understanding of the course material. He has an issue with handing in work on time and has had to hand in a second term paper in one of the classes, as the first one was a gross plagiarism. He particularly requested that I give him this letter of recommendation, although I twice said that I didn't think that this would be a good idea. We will be glad to see him graduate, just to get him out of our hair.
EuropeanFemaleScienceProfessor

Letter of Reference Contest Results

Below is the unanimous choice of the FSP Editorial Board for our favorite of the Letter of Reference Contest submissions. The letter, written by Wendy Panero, has some of the classic elements of the LoR genre: a strong recommendation that is not backed up by anything in particular in the text of the letter, which was written based on little information.

To Whom It Might Possibly Concern:

I hope you accept the letter even through it comes two weeks after your deadline.

I would like to give Susie Student my highest possible recommendation for Your Program. She told me yesterday she was planning to apply, and since she begged me and said she gave me a really good teaching review, I agreed to write the letter.

Susie took my class, Introductory Rock Appreciation, this quarter. She earned a C, but I gave her a B- when she told me that the only reason she did so badly on the final was that her grandma died. Susie is among the top half of the students I have encountered here in my two quarters teaching at Massive State University, so I really would have enjoyed seeing Susie’s full potential in this class. She was working as a dog sitter for much of the quarter, however, so that took her attention from rock appreciation. While clearly disappointed that the course addressed actual rocks rather than the music she expected when she enrolled in the class, she did attend class most days.

I know Susie Student is an excellent match for Your Program.

Signed,

Super Junior Faculty Member
Department of Inferiority Studies
Massive State University
Middle of the Country, USA


more tomorrow.. and the next day..