Teaching evaluations are best viewed in a general, cosmic, slightly unfocused way as an indicator of some aspects of a teacher's methods and effectiveness. I just got my spring evaluations yesterday, and I am pleased with the results, but that won't stop me from musing about the bad (and good) aspects of evaluations.
I think that if you look at someone's teaching evaluations for a range of classes (small and large, majors and non-majors, graduate and undergraduate) over 3-5 years, you will get a sense for whether that person is a 'good' teacher or not. Ideally, 'good' means interesting, caring, challenging, accessible, organized, and fair, and doesn't correlate with a propensity to give A's for mediocre work and effort.
At my previous university, I had 2 teaching mentors who sat in on my classes at random times during the semester and provided me with feedback. They were supposed to tell me in advance when they would be sitting in, but they didn't. This was fine with me because it meant they saw the class more as it was on a routine day, rather than on a day when I had prepared in a different way knowing I'd be monitored. I didn't really get any useful feedback from these teaching mentors except about some technical issues on how to more effectively use the A/V equipment in the classroom. Even so, I liked the fact that there was an additional evaluation method of my teaching other than just the student evaluations.
At my present university, our only feedback comes from the teaching evaluations that students fill out at the end of each semester, so these evaluations are the sole evidence for whether we are good teachers or not. That's why it's important that the evaluation process be improved.
Problems with the current process (written from the perspective of someone teaching at a large university):
1. Some of the questions are strange, irrelevant, offensive: I have written about this before, but some of the primary questions that are the basis for the quantitative aspect of the evaluations are inappropriate, e.g.,
- A question about the physical environment of the classroom asks students to rate me on something over which I have no control.
- A question about how the students would rate the professor's knowledge of the course material is offensive as written. That question could be worded to ask how effective the professor is at conveying their knowledge to students, rather than asking the student how much they think the professor knows.
- A question about how much a student learned in the course is not only rating the effectiveness of the teacher, but also how much effort the student put into the course. Sometimes the two are related (students may put more effort into an interesting and well-taught class), but sometimes they are not. A student who does not attend class regularly and doesn't do assignments etc. will not learn much in the course, yet he/she gets to rate how much they learned in the course as part of the evaluation of the professor's teaching. In theory, these students are a small minority and their low ratings will be outliers in an overall positive evaluation, but their effect on the evaluation depends on class size.
The questions I think are good and useful include:
- A very general question on the professor's overall teaching ability.
- A question or questions about the professor's level of respect, concern, accessibility for students.
- A question about whether graded work is returned in a timely way.
Even for those questions, a student who is not doing well in a course will in some cases give a professor a low rating, but again, if the class size is sufficiently large, these will be outliers.
And then there are the detailed comments. Reading these can be alternately heart-warming and enraging. I am fortunate that this semester I got the heart-warming kind. More typically there are at least a few that will say "I wish you had done X", where X is something unreasonable (e.g., "I wish you could have given us all the answers to the quiz questions in advance so that we could have studied them.").
And of course there is no chance for rebuttal/response, and there is no way to fix any problems for that specific class. That can be frustrating.
In summary: it's a flawed process that sort of works in a general way.
9 years ago