To get everyone in the mood for the theme of this week: Letters of Reference (for graduate school applications), consider first the dreaded Ranking Form that accompanies many such Letters of Reference.
Starting tomorrow I will post the results of the Letter of Reference contest.
**********
REFEREE FORM
Please rate the applicant to the best of your ability, making it clear what your frame of reference is (examples: all students you have ever known; students who took a certain course that you have taught at least 7 times; students who fit certain arbitrary biophysical parameters that you believe are significant even if this indicates something quite negative about you, not the candidate; etc.). In the space below, please also indicate how long and in what capacity you have known the candidate.
In your numerical rating, use the following categories:
Top 1%, top 2.5%, top 5%, top 8%, top 8.5%, top 10%, top 25%, top 50%, lower 50%, lower 25%, lowest of the low and/or scum of Earth.
Research ability:
Intellectual ability:
Scholarly capacity:
Ability to form thoughts:
Creative skills:
Capacity for innovative work and/or thought:
Potential for deep thought:
Ability to work independently but not too independently:
Ability to work well with others:
Ability to work well with a diverse group of people:
Social skills in a variety of academic settings and pubs:
Ability to express ideas in spoken form:
Ability to express ideas in written form:
Ability to express ideas using appropriate images and color schemes:
Clarity of thought:
Motivation:
Depth of commitment to chosen field of study:
Level of focus:
Depth of focus:
Degree of focus:
Quantitative skills:
Qualitative skills:
Analytical skills:
Graphical skills:
Ethical standards:
Integrity:
Morals:
Leadership:
Followership:
Maturity:
Emotional stability:
Knowledge in chosen field of study:
Knowledge of world geography:
Can he/she spell?
Potential to be a good teacher/TA:
Overall recommendation:
I teach undergrads in a college in India. I thought the problems I had in filling these forms accompanying letters of reference was because I was not used to your academic system. I am glad to see it's not just I who am intimidated by these...I am never sure...perhaps so and so was better than this student ... perhaps this student will turn out to be bad at research-after all an Indian undergrad does next to no research....so on... keeps going around in my mind days after I send it off.
ReplyDeleteI have a pile of these to do. When I agree to write references I want the students to get what they are applying for. It's becoming an art form -- what to say to give the student an edge -- inflated letters to go along with inflated grades.
ReplyDeleteDoes the candidate drink beer and own a car?
ReplyDeleteI'm slightly autistic and if that's anything to go by I wouldn't fit into academia at all, or any part of this world for that matter....but I'm still here, I must be doing something right...
ReplyDeleteUnfortunately, my workplace is moving in this direction with employee performance evals in order to “standardize” them across the lab. If I didn’t think that it would really piss off one of my bosses, I would suggest that they add the following question to our evals:
ReplyDeleteThis employee possesses a certain je ne sais quoi: (a) strongly agree, (b) agree, (c) somewhat agree, (d) disagree, (e) I don’t speak French.
Seriously though, I have always wondered how admissions committees interpret these forms. Are one’s percentile scores similar to a GPA, as in, it would be nice to have a 4.0 but anything above a 3.7 (top 20%?) should be fine? If the recommender writes a substantive letter, does this increase/decrease the weight placed on their percentile scores? What if the letter is very good but the percentile scores for some reason are not? Are scores averaged across multiple recommenders?
Finally, I think the ability of a recommender to judge an applicant’s “motivation” is about as accurate as a student’s ability to judge a prof’s interest in being a good teacher.
Hilarious. As an aside, world geography seems like an underappreciated difficult subject. Contestants on several reality shows/game shows that I've seen recently seem unaware that a) Europe is a continent, not a country; b) Prague is a city, not a country; c) Africa is a continent, not a country, etc. etc. It makes for very entertaining TV, but doesn't speak well of our educational system...
ReplyDeleteYay! Best time of the year!
ReplyDeleteLove the percentage rankings. I can hardly wait to see the reference letters.
I'm in the midst of doing recommendations now, and particularly liked the school which asked me to divvy up the Exceptional category (top 5%) into subcategories of top 1%, top 2-3% and top 4-5%. There are only a few circumstances where I can even imagine this is possible. To those who review recommendations, does anyone even pay attention to these? I think my rankings may say more about me (inability to rank anyone exceptional) than about the student. Is the applicant punished when I don't just go through and consistently rank them exceptional.
ReplyDeleteI am so excited! Can't wait to see the results of the LOR competition!
ReplyDeleteI have friends in another lab whose advisor is a Nobel laureate, so their letters of recommendation should count for more? (I could probably finagle a LoR from said Nobel laureate too via my friends, if I bought him a bottle of wine or something!)
ReplyDeleteHaving just read 20 admissions files, I can say that I totally ignore the forced-choice ratings. I have no idea what the calibration is of the person clicking the radio buttons, and they are not in anyway correlated with what I want to know about the applicants.
ReplyDeleteI rely mostly on substantive letters---ones that talk about the research that the applicant has done with the letter writer.
I've read tons of admissions files. I always ignore the forced-choice rankings, and just read the letter.
ReplyDeleteI've written tons of recommendation letters. I hate the schools that impose their own forced-choice rankings, instead of just accepting a PDF with my letter. I tell my assistant to make up random answers to the forced-choice rankings, or to assign the very best possible score in every category to every applicant. They're total nonsense and I feel like I'm doing the world a service by randomizing my values on those forms.