Thursday, January 15, 2009

His or Hers

If regular visitors to this blog are starting to get the impression that I spend a lot of time reading letters of reference and pondering their myriad intricacies, that's because I spend a lot of time reading letters of reference and being amazed at what I read.

Today I am thinking about the unfortunate situation in which an apparently talented student had only 1 of 3 reference letter writers who took time and care with their reference letter. The other 2 wrote brief and uninformative letters.

I can understand writing a brief and uninformative letter if your only interaction with the student has been in a lecture-based course. Some students ask me for letters and I tell them that I would not be the best choice because I can't say much more than what is apparent on the transcript, e.g. Bob got an A in my class last semester. But sometimes they have no choice because that has been the extent of their interaction with all or most of their professors.

I can bulk up a letter a bit by talking about how rigorous my course was and how only 2 students got A's in the class and Bob was one of those. I can say that Bob asked insightful questions in class. But that's still just a paragraph.

So I can kind of sympathize with one of the short letters, which was written by someone who probably only had the student in one class and didn't have a lot more to say than what is in that paragraph.

The other letter, however, annoyed me because I think it is a form letter. There are various form-letteresque aspects of it, but what caught my eye (and that of other faculty reading the letter) is the use of the pronoun "his" in part of the letter. The applicant is a "her".

It is possible that the letter-writer just slipped up because 94% of the letters he writes are for male students and he didn't check over his letter and he was really tired because he had to write 37 letters of reference for students who only gave him a few days notice before their urgent deadlines and he didn't know this particular student all that well and the head gasket in his car needs replacing and that is expensive and his cat may be suffering from seasonal affective disorder.

I can try to not be so annoyed with the careless and uninformative letter-writer. But I will fail. I will fail because these letters are important. This student's chances of admissions to a graduate program will likely not be harmed, but in a competitive pool, an applicant with impressive letters may prevail over an applicant with vague letters. An applicant with impressive letters may also be at an advantage for fellowships and other funding.

Fortunately in this particular case, the applicant had a successful research experience with one faculty; the one who wrote the most detailed letter. One really good and detailed letter goes a long way towards making up for the other, uninformative letters. Also, admissions committees understand that not all letter writers are conscientious and/or nice people and this is not the applicant's fault.

In general, letters are not that useful anyway, but we still need to see them just in case. Ideally, what we see will be an honest and thoughtful appraisal, not a form letter.

33 comments:

usagibrian said...

...I can't say much more than what is apparent on the transcript, e.g. Bob got an A in my class last semester.

FERPA Mode Engaged...

In point of fact, unless Bob gives you permission to disclose his grade, doing so to any third party, even in a reference letter, is a violation of his privacy under FERPA. A general statement that he waives his FERPA privacy rights is sufficient but necessary to discuss his academic performance in a reference letter.

In an ideal world, students requesting a letter would provide a signed release from the school they're applying to that would cover the allowable areas of disclosure and provide some guidance as to which areas the school is interested in (and whether the student is waiving access to the letter--remember if Bob enrolls at the other school, he may view the contents of his admissions file under the records access provisions of FERPA).

In the absence of waiver provided by the school he's applying to, I'd suggest obtaining a minimum of: "I, Bob, give permission for you to disclose specifics of my academic record protected by FERPA in this reference letter as you deem appropriate." Without that release, brief and uninformative is all you've got to work with because there's virtually nothing about the student's performance you can legally discuss.

Anonymous said...

Three letters is just too many at that level. For graduate admissions we only require two, and really only one of them is usually technical (from a professor or lecturer who knew the student well) and the other is from the academic advisor equivalent (they know the students better here than they do in the states!)

Alyssa said...

I know this will never happen, but I personally think that the etiquette of reference letters should change; the applicant should be able to read the letter before deciding to use it.

There was a situation in our department where one of the professors was writing horrendous letters for everyone, including his own graduate students - even if he sang their praises in the halls.

Even if it's not a bad reference, but just a poorly written one (like you discuss) - the applicant should be able to decide for themselves if it will help (or hurt) them.

In my particular experience, I had a hard time getting scholarships my first two years of my PhD. Then I switched references and all of a sudden I received two big scholarships. This led me to believe there was something the other reference was saying that didn't put me in a good light (or the letter was poorly written). I'll never know though, since I was never allowed to read the letters.

Anonymous said...

I just reviewed an applicant who got an obvious form-type letter from Big Famous Scientist, but then I noticed there was no letter from the person he worked for in research over 2 years. So, either this kid is stupid and thought BFS would be more helpful than a detailed letter from someone with extensive interaction with him.... or, there is some reason he couldn't get that letter. Making it tough to judge him fairly since otherwise his application is pretty strong. So, for anybody reading this blog who is applying - use people who know you!!!

Anonymous said...

I thought letters were one of the most important things in a grad school application since that's where you really get to "know" the applicant.

The GRE is only good to filter out the very low scores (say low than 600,500 or so), but at the end a 700 or a 900 in a subject test means nothing in terms of how good they will be at research. Just look at the average score for US vs. Foreign students in the US. I found at least two institutions that report this for Physics (Colorado at Boulder, http://www.aip.org/gpb/pdf_files/091.pdf, and Univ of Texas at Austin http://www.ph.utexas.edu/grad_admit.html)

The only evaluators left are letters and the statement of purpose and thus I thought they were the most important two. Interesting to know that letter might not matter much.

Anonymous said...

1. Letters do matter a lot! However, they need to be from someone who knows the student. The best are those that come from someone with whom the student did research,as they can be detailed and relevant. if a professor is doing a poor job at writing a letter for someone with whom one worked closely, that is unforgiveable. A detailed letter from a research advisor is worth ten letters from a professor who had the student in a class. If a student has no research experience, that comes out in the letters in both ways, and is a major deficit that needs to be corrected before trying grad school.

2. However, all letters I write for my classes are to some extent form letters, as FSP herself acknowledges. I certainly do not draft a letter from scratch for my 150 student class. I do TRY to make sure I change all her's to him's and he's to she's, but if I miss one... You must not have had enough coffee this morning, FSP--you're altogether too grumpy about this. I do, for my smaller class where I actually get to know the students, try to personalize it, and often can write a pretty strong letter for those students. Expecting undergraduates to get three letters from someone who knows their potential for science in detail is expecting WAY too much.

Mark P

Anonymous said...

I once received a reference letter from a job where I had two bosses (and both submitted letters). One was fine but the other was not only brief, but had poor sentence structure and misspellings. I didn't know what to do in that situation (ask them to fix it? Try to find another letter?).

Female Science Professor said...

I am grumpy about the his/her/form- letter thing because it won't get past a fellowship committee that has to make difficult decisions about a group of outstanding applications. It may not be fair to the student to judge their reference letter writers' ability to write compelling, personalized letters, but the committee has nothing else to go by.

Anonymous said...

FERPA:

I am not a lawyer, but isn't the student implicitly allowing you to release this information by asking for the reference letter?

Say, if I ask a doctor for a second opinion, I don't have to explicitly grant him access to my medical file. The request for the opinion implicitly establishes this.

Anonymous said...

I would never use a form letter for a student I know well or for an application to grad school. Most of the letters I write, though, are for premeds whom I don't know at all. These letters go to our school's premed office where someone reads them, sorts out the good from the bad, and then puts together a composite letter incorporating glowing statements gleaned from all the letters. I don't know how common this practice is - I know my grad university did it as well - but it offends me and I always send form letters to them on principle.

FSP, you are not the only grumpy one today.

Anonymous said...

I have done worse than that: When I was applying for faculty jobs, I got school names mixed up in a cut-and-paste job. Everything else in the cover letter was specific to the school, mentioning facilities and people I'd like to work with, but I pasted the wrong name in one part. Amateur mistake, and I don't blame them one bit for not interviewing me. Still, while I got what I deserved, it's hard for me to get too upset at busy people who make similar mistakes when sending out a bunch of letters at once. (Although on him/her, surely a "search/replace all" isn't that time-consuming for a form letter!)

I have a very good student that I wrote a custom letter for, but when I customized it for different schools ("He would be an especially good fit for your facilities using blah blah blah...") I got paranoid about cutting and pasting the right names into the letters. I'm pretty sure that I didn't make any mistakes there....pretty sure....

Say, if I ask a doctor for a second opinion, I don't have to explicitly grant him access to my medical file. The request for the opinion implicitly establishes this.

If you go to the second doctor, somewhere in the forms you sign for the visit there will be a release of medical records form. Or some language about it on the treatment consent form. They are far more experienced than professors when it comes to covering themselves on the law.

Anonymous said...

I'd be much more forgiving of a him/her mistake than you seem to be. In the old days -- 5 or 6 years ago -- a student would ask for a letter, I'd draft one, and hand it off to an admin to proofread (for such dumb mistakes), copy, and send via envelopes prepared by the student.

These days, it's all e-mail, so I have to handle each individual letter individually, often with some accompanying web form. It's as much effort making sure the admin processes everything as doing it myself, so I generally do it myself. And I'm sure mistakes happen.

Also, most professors don't like writing weak, form-type letters. But I've had students who I've tried to dissuade from having me write a letter (because that's about all I could write) insist that they really didn't have other professors who could. In some sense, weak letters like that are informative -- they show how much varied experience a student has had.

Anonymous said...

Would love to hear some comments on the following awkward letter situation:

I have been asked to write a rec letter for a fellowship for a student who is in the lab of a close colleague. I know the student well and we have collaborated with the student and his mentor previously. However, I cut off the collaboration (tactfully) because the student was so scientifically irresponsible that we couldn't trust any of the data.

I had previously expressed concern about this student to his mentor, so I was a bit surprised to be asked for a letter. Normally, if I can't give someone a good letter, I don't agree to provide one. However, in this case, I feel like I may offend my colleague by doing so. The situation is exacerbated by the fact that the fellowship proposal itself is poorly written both at the grammatical and conceptual level. I never write really bad letters, always honestly pointing out the student's strong attributes where possible, but I think readers can clearly sense the omissions. In this case though, I can't think of much good to say. What would you do?

Anonymous said...

I generally don't say any bad things, usually relying on omissions for the reader to glance the weaknesses or I simply politely decline to write the letter. However for truly egregious cases I believe that professional ethics demands that one must broach the subject in a sensitive way, e.g. "I had an ongoing collaboration with the student but some of the methods he used to collect the data left me uneasy, so I suspended the collaboration".

Notice that you are not accusing the student of anything, but simply describing the facts as they transpired. *You* felt uneasy, *you* suspended the collaboration.

daisy mae said...

it's the student's responsibility to get the forms to the people they wish to have write their letters within a reasonable amount of time. that's what i did - and also met with each person to talk about the schools i was applying to - what i liked, why i wanted to go there, and what my goals were in pursuing my PhD.

although i didn't do any "real" research (just loads of lab-based courses) and never published, i was awarded a top fellowship and told my the committee that i was their top choice BECAUSE of my reference letters...

i also spent the time making sure that the three people i asked 1) knew me, my story, and my work ethic well and 2) could write and express themselves well

plus, students DO have the right to read the reference letters - but they are also given the right to waive that option by checking a box on the form sent out/required by the institution (at least in the states).

Anonymous said...

There are several comments above about how many letters of recommendation schools should ask for - that three is too many because surely no student could know that many professors.

I think the purpose of asking for multiple letters is to determine the extent of a student's research experiences. Students with multiple research experiences with different people/labs will come to graduate school with a broader skill set than those who had for example one excellent REU experience. Multiple letters allow the students who have REALLY pushed to do the most research to shine.

I'm not saying that someone who only has one "research letter" is deficient, simply that when compared with someone who has shown they can work well with multiple people on multiple projects is a stronger candidate.

Also, if you are an undergrad who loves your lab and would like to stick with it, I'd think it would be a good idea to try to find ways to work with several people in the lab - post docs, graduate students etc. Each person you work directly with can then write you an individualized letter, detailing their specific experiences working with you. And the PI will write a broad letter outlining your entire experience. I think such a package is much stronger than Research PI + undergrad adviser + Prof you took one upper level course with...

Anonymous said...

Say, if I ask a doctor for a second opinion, I don't have to explicitly grant him access to my medical file. The request for the opinion implicitly establishes this.
____

You actually do, now that we have HIPAA- I have had to do this. Also, for my research we do MRIs of people, and frequently they'd like a copy of the picture of their brain. They literally have to sign a release to allow us to release the picture of their own brain to themselves!

usagibrian said...

I am not a lawyer, but isn't the student implicitly allowing you to release this information by asking for the reference letter?

FERPA Mode Reengage

No. FERPA is inherently non-implicit. Yes, it's a pain. That's the point. It's a feature, not a bug.

The guidance from AACRAO's FERPA Guide states: "Statements made by a person making a recommendation that are made from the person's personal observation of knowledge do not require a written release from the student who is the subject of the recommendation. However, if personally identifiable information obtained from a student's education record is included in a letter of recommendation (grades, GPA, etc.) the writer is required to obtain a signed release from the student which (1) specifies the records that may be disclosed, (2) states the purpose of the disclosure, and (3) identifies the party or class of parties to whom the disclosure can be made."

It is also worth noting, if you as the recommending faculty member retain a copy of the letter for your records, this copy becomes part of the student's education records at your institution and subject to review under FERPA. This doesn't attach at the destination institution unless the student enrolls, but the copy that resides in your file must be made available to the student for review upon request (unless the student has waived access and you retain that waiver).

Anonymous said...

"The guidance from AACRAO's FERPA Guide states: "Statements made by a person making a recommendation that are made from the person's personal observation of knowledge do not require a written release from the student who is the subject of the recommendation. However, if personally identifiable information obtained from a student's education record is included in a letter of recommendation (grades, GPA, etc.) the writer is required to obtain a signed release from the student which (1) specifies the records that may be disclosed, (2) states the purpose of the disclosure, and (3) identifies the party or class of parties to whom the disclosure can be made.""

The summary on the Department of Education page actually lists the following:

However, FERPA allows schools to disclose those records, without consent, to the following parties or under the following conditions (34 CFR § 99.31):

* School officials with legitimate educational interest;

I think it's an accurate assessment to state that all graduate admissions fall under that category, the first listed. It seems pretty well covered under § 99.31(a)(2) for that matter.

I think we're all pretty much on board with the implicit OR explicit status of applications to new schools and the disclosure of information as part of that petitioning effort.

Anonymous said...

Anon at 1:27 -- that's hilarious!

Regarding FERPA: does anyone follow all these guidelines? I don't think I know anyone who does. What are the possible repercussions of not following them? Would a student have to file a complaint to get the legal ball rolling?

Anonymous said...

It seems to me that the information provided by anon 3:40pm settles it and one is allowed to release grades provided the recommendation letter is for admission to grad school and not for a job (academic or otherwise).

Anonymous said...

Usagibrian,

Stop causing problems by misinterpreting the law. It is allowed under FERPA to release information without prior consent for grad school reference letters.

Anonymous said...

To Alex the cut-and-paster: I just served on the graduate admissions committee in a top university. The best applicants in our pool applied to many universities -- this is wise behavior and shows that they understand the way things work (competition for the few available slots is fierce and acceptance can't be guaranteed, however good you are). Some applied to more than 10 universities.

Certainly I didn't expect their recommendation letters to be tailored to each of the universities in question. After all, we want to know why the student is great for us, not why we are great for the student. Many of the letters said "it is my pleasure to recommend XX for admission in your program" without any reference to which university this program is in (or even the title of the program). No one in the committee ever complained about this.

We did complain about devoid-of-content letters from top researchers, but the type of 'form' where the letter is specifically about the student in question, but not specific to the university s/he is applying to, was not a problem. This may be peculiar to our committee, but I would guess not.

Finally, re the edible thank-you notes: a colleague of mine received as a 'thank you' a card and an iTunes gift card for a sizable amount of money. Is it only me, or do others also think that this kind of 'thank you' comes too close to a bribe and should be absolutely discouraged?

Female Science Professor said...

I don't mind a form letter if it is specific to the student but not the university. I don't like form letters that could be for any student, with no effort to personalize the letter other than changing the name at the top of the letter.

EliRabett said...

Let me put this together with your previous post. We have a few position for summer research. What we have found best is to approach someone we trust at a place we want to recruit from and tell them that we want THEM to choose the summer research student.

This has a lot of advantages. The person we approach knows that her reputation is on the line. We build a relationship with her and her department and we get a good research student for the summer.

usagibrian said...

It seems pretty well covered under § 99.31(a)(2) for that matter.

Yes it does. "(2) The disclosure is, subject to the requirements of § 99.34, to officials of another school, school system, or institution of postsecondary education where the student seeks or intends to enroll." Looks like a 2005 addition. Time to update my guide.

Anonymous said... Usagibrian,
Stop causing problems by misinterpreting the law. It is allowed under FERPA to release information without prior consent for grad school reference letters.


Take it up with the Department of Education and AACRAO. The guidance I'd received, including conference sessions with representatives of the Department last year, said otherwise. I freely admit, I'm very conservative in my interpretation of FERPA, but I'm nowhere near as conservative as some. I stand by the advice that getting the student to acknowledge what may appear in the letter is still a good idea even if it's no longer part of the regulations. It may also prompt faculty who had their last FERPA training session in the 90s to be more forthcoming.

amy said...
Regarding FERPA: does anyone follow all these guidelines?


Probably not. It's a massive, confusing, and occasionally contradictory law with great swaths of undefined terms. That's no excuse not to make a good faith effort to follow it.

What are the possible repercussions of not following them?

The ultimate: the institution may lose eligibility to participate in the Federal Title IV Financial Aid programs. That's not going to happen over a reference letter, but it is a possible sanction. That's why I err on the side of caution.

FSP: Sorry about hijacking your thread. The point I missed making was, does the department provide a template for the student to give the reference writer? Our admissions department has several specific, non-academically oriented questions they ask be addressed in the reference request, and they usually get responses that address the points they're interested in.

Anonymous said...

In my experience*, letters are the most important part of an application. We have found that the student's research experience is the best predictor of success in graduate school. (Better than good grades in appropriate classwork, which is better than GREs, overall GPA, or school of origin.)

The problem we find is students who look good but have meaningless letters (form letters, missing letters from
person they did research with, letters from graduate students rather than faculty, etc).

Three letters for application to graduate school or an NRSA is very important because it allows the student to survive one bad letter writer. If one of the letters doesn't match the other two, we ask ourselves why. Sometimes the answer is clear - that the writer knows something about the student the others don't or that the writer is a jerk. Sometimes we can't tell. But it causes much discussion.

Also, no one on any of these committees have ever been seriously concerned about mis-labeled universities. ("This student is a perfect candidate for University X" when we're university Y.) We usually have a good laugh. What we really look for is detail about the interaction with the student. Form letters, like the one FSP is describing count significantly against the student because it implies that the student either does not have faculty to write letters or has made a poor choice about letter writers. It is possible to survive this if the other letter writers are particularly good.

* experience = being on multiple graduate program admissions committees at research-oriented state-U, chair of my main program's admissions committee this year, serving on NRSA study sections, as well as faculty search committees.

Anonymous said...

The contrast in trust is interesting. STUDENTS need to have letters written by someone who knows them well. INSTITUTIONS can accept and trust letters from someone they have never heard of.

I guess the rationale is that the student stands to benefit more from the chance to attend the institution than vice versa? To the institution there are lots of fish in the sea?

Anonymous said...

I wish they would personalize rejection letters. "We did not like you because we heard that you ate too many beans..."

John Vidale said...

I just ranked about 30 applications for our sub-discipline last week, aiming to admit 6 and get 3 for next fall.

Letters are paramount. Scores and grades should clear a threshold, statements should be spelled correctly and articulate, but the bottom line is whether one or more letter writers can attest to performance in undergrad RESEARCH projects. Showing one good project is good, two is excellent.

The only trump to stellar letters is making a deep impression in personally presenting research at our field's fall meeting.

John Vidale said...

One more detail - awash in paperwork as we all are, I can't imagine either

(1) getting a student to do legal paperwork to be able to reveal their grade in a letter or

(2) worrying about some bureaucrat coming after me for revealing a grade in a letter of recommendation without the paperwork.

I saw dozens of such grade discussions in files this year, from people who I expect share my dislike of frivolous paperwork.

Anonymous said...

On the topic of reference letters (but slightly offtopic in terms of who writes them):

I teach at a Canadian university but am and was raised British. I have a lot of interaction with the first and second year science students and have therefore started to be asked for reference letters for various scholarships, medical school applications, etc. Being a Brit in Canada, I suspect my writing style is somewhat less effusive than I understand is the custom in the US - though I have yet to write a letter that I didn't consider glowing. My letters have so far always included very specific information about why I felt that particular student was excellent. I'm fortunate that all who have requested letters thus far have genuinely been excellent. (I suspect the students who only did averagely in classes in my field got their letters from instructors in their other fields).

When writing letters for US medical schools, should I be making an effort to be extra-effusive or do the admissions committees already understand that letters from other countries tend to be less exaggerated than what I understand is standard practice in the US? I'm really hoping they do, as I don't personally feel comfortable claiming a student is better than they are. For example, I would be really really uncomfortable making statements like "X is the best/smartest/hardest working student I've ever taught" since it would be unlikely to be true - and certainly couldn't be used on a regular basis.

!=42 said...

you can tell from the headlines of the blogger (woman science professor etc.) how megalomaniac she is. i am a phd student at a "top ten" engineering school in the u.s. and in the past three years i've seen quite a few such items like this blogger. they live like parasites on their students. they work no more than eight hours per day themselves, and expect their students to do everything for them - from writing grants to reviewing papers. oh yes, the papers in the big shot journals which are supposed to be reviewed by professors are often done by their students. also, they do not get actively involved in the research and make stupid comments during the weekly meetings.
i really can't understand why someone should expect his/her students to work for 40 hours when it is clearly written on the offer letter that the grad student is expected to work for 20 hours against the stipend he is paid. this is plain exploitation. i think a student of such an adviser should make a formal complaint against the professor to the university or the human rights organizations when they leave the university.
and my dear professor, don't think you are doing your student a favor by inducting him/her into the university or paying his tuition. these students are often as smart as you - and hey, you wouldn't get any money from the grant yourself unless you show some student working on the project. so you actually need the student for your own sustenance.
of course, all professors are not like this blogger. they work symbiotically with their students (expecting reasonable working hours from them) and make significant contributions to science.