Room for Improvement

Student comments on my teaching of a particular course:

Great professor!
I have enjoyed this class!
I liked the readings.
This course required too much previous knowledge.
Professor very helpful with homework.
Homework very useful for class.
Well-constructed lectures.
Very organized lectures.
She speaks very clearly.
She answered my homework questions.
She provided images and charts to supplement the subject matter.
The in-class exercises were helpful.
I liked the practice exercises we did in groups during lecture.
I liked that she asked questions during class and this helped deepen my understanding of concepts.
Useful supplementary material to help us understand lecture material.
She explained the topics completely in class. Didn't use a textbook as a crutch.
It was great that lecture and lab material were well coordinated.
She was always ready to answer questions.
She was always willing to help with any questions.
She provided the subject matter very clearly.
The last project was too much work for this level of class.
Lecture presentations very clear.
I liked the in-class exercises.
You should improve your teaching methods.

Note that almost all of the comments are in the 3rd person (except for the last one), as if the students were writing to someone else about me, rather than writing to me with feedback. I don't know if it matters in terms of type and level of feedback whether the student is imaging an unknown audience or speaking directly to me (?). At evaluation time, I give a little talk to the class about the importance of this feedback and how it is used by instructors and the department/college/university, but I think there is still general confusion among students about what exactly the purpose of these evaluations is and who reads them and whether anyone cares what they think.

These are overall nice comments, and unfortunately also rather classic in that the criticisms are too vague to help me understand what the specific complaints are.

The last comment, despite being too vague to be useful in any specific way, is absolutely right. Despite being deep into my mid-career years, I don't want my teaching to fossilize. I want to improve. In recent years I have attended teaching workshops and gotten some ideas from those. When I team-teach, a faculty colleague is in the classroom with me, so I get some peer feedback. And last term, I jettisoned the too-long and too-detailed textbook and provided focused readings, including some that I wrote myself. That seems to have worked quite well (or at least no one said they missed having a textbook), so perhaps that counts as an improvement. I would also like to do some new things involving e-learning and have been to some workshops and meetings about that.

I am thinking about teaching because I was just looking at my evaluations, though mostly I am enjoying having lots of uninterrupted time for research. This week I even managed to submit a manuscript on which I am primary author. It's been about two years since I've been able to do that (and I don't mean to imply that I did it alone -- an excellent colleague was essential to the completion of this paper).

As I was finishing the paper (and a related grant proposal) recently, it occurred to me that I could create a new teaching module based on this work and incorporate it into the class for which I just received teaching evaluations (not, of course, as extra work but replacing some older material). Probably more than any major change in teaching style, a realistic way that I can improve my teaching is to find good ways to incorporate new material -- specifically, integrating New Science with Classic Science, so that students learn the fundamental stuff without which they are incomplete as scientists and people and yet are also exposed to new things that help them see where the field is at (including being exposed to unresolved questions that might inspire them).

Anyway, it's been a busy summer so far. My father recently asked me if my husband "also has the summer off" and I was actually quite calm about it this time. Have you had a similar conversation with anyone yet this summer? Parents? Neighbors? Friends? Students? Assuming that you do in fact work in the summer even if you are not teaching, did you (1) smile serenely and let them continue to exist in ignorance; (2) correct them (a) calmly, (b) not calmly; or (3) lapse into stony silence (if having a conversation) or send a glaring emoticon (if in e-contact)? (or other..).

Complaining Early & Often

While serving on a particular review committee at my university, I have seen many examples of negative comments in teaching evaluations for problems that seem like they could easily have been fixed during the course if only the professor knew there was a problem (and was willing to recognize it as such and change something about their teaching style).

Sometimes we professors can sense that something is wrong or that students are unhappy or confused. Some students will tell you directly, but most try to express their unhappiness and/or frustration in unspoken ways. Unless you ask them what the problem is, perhaps even by doing a mid-term evaluation to get anonymous comments, it can be hard to know what the problem is in some cases. Of course, if you just handed back a test and the average score was 17%, you might have some clues as to why students are unhappy.

The situations I am thinking about don't have to do with difficult tests, but more with teaching style. I have talked about some of these issues before, such as pacing vs. being stationary, having an accent and/or speaking too fast, using various formats and devices for writing, projecting etc.

Whenever I see a file with very negative teaching evaluations, I always wonder if any students complained somehow, to someone, during the course. Some of the problems seem entirely fixable during the term, when there is time for the students to benefit.

I tend to assume that no one complained because I seldom see a comment like "Although we told Professor X that we could never see his writing when he used the red marker, he kept using it." Instead, it's more common to see the complaint "I could never see the writing on the board when he used the red pen."

I can certainly understand why some students would not want to complain directly to or about a professor during a class. What if the professor gets angry and punishes them by giving impossible exams and low grades? There is surely some anxiety about the consequences of complaining.

Of course, there is a difference (or can be, anyway) between complaining and making a request. That is, instead of "I hate it when you use the red pen", something along the lines of "It would help us all see your writing on the board better if you only used the black and blue pens."

Other problems, of course, are more serious and more difficult to fix during a course; for example: comments about a professor's disorganization, inconsistency, perceptions of unfair tests or rude comments, refusal or inability to provide clear explanations or answers to questions. In those cases, what can a student do?

Don't wait for the teaching evaluation and don't be satisfied with writing negative comments on some professor-rating website. Get organized: talk to other students, find out what the issues are, get examples, and write it all down. Then ask an undergrad advisor, respected senior professor, or relevant administrator what to do. If the complaints/requests are reasonable, perhaps there are faculty or administrators who can pass along suggestions aimed at fixing problems in time to help the students. In some cases and at some places, these concerns will be dismissed or ignored, but I think it's worth a try.

In some cases (but probably not the extreme ones), there might even be a reasonable explanation for what seems like bad teaching. I one case I can think of, a professor used a lot of text-heavy slides in a class. The students complained about it in the teaching evaluations, but not one of them had mentioned during the course that they hated this. It turns out that there was a hearing-impaired student in the class, and the professor had been asked to put a lot of text on slides, and had spent considerable time doing so, out of concern for the hearing-impaired student.

There are ways that this particular situation could have been dealt with better by the professor and the students. For example, the professor could have explained what was going on, and could have found a way to present the course material without making the students feel bludgeoned by text-laden slides.

Of course I wish the major teaching problems could be fixed, but it is these easily fixable problems that I am focusing on today because they are fixable with a bit of two-way communication between professors and students.

So, student-readers: Are there any possibly-fixable issues in the classes you are taking now that you wish could be fixed during this term? If you give us some examples, the professor-readers can comment and say "Yes, you should definitely tell your professor about that" or "No, don't do it" (but here's a suggestion for dealing with it). More likely, you will get both answers for any particular example, but the results could be interesting anyway.

Will There Be Anything Else?

One of my colleagues has a teaching philosophy that contains the following principle:

If you do a lot for your students in terms of providing 'extra' materials (study guides, lecture notes etc.), they will ask for more and more and more and be unhappy. If you don't provide them with much -- maybe just some review materials before an exam -- they will be content with what little you give them.

This is a bit cynical, so I hasten to note that my colleague is a great teacher. When he teaches a large intro science class, students applaud at the end of the last lecture. Sometimes when he is in a near-campus coffee shop, former students have the barista send drinks and treats to his table with their thanks. Students know he cares about the course, even if he pours most of his energy into the in-class part and not so much to other parts.

I have always disagreed with him about the "give them an inch.." hypothesis, but I must admit that every once in a while there are some data (or a datum) to support it.

At the moment, I am thinking about a comment on my recent teaching evaluations.

First consider this: After each class, I post questions that cover all the main topics of that day's lecture and I provide a pdf of the presentation file if I showed images during class. During the term, I give the students all the exams from the previous year's class. And, in a new development for me this past term, just before class time I post a file with the images I will show in class that day. I annotate the images in class, and the students who bring electronic devices to class can do so as well if they want. For those who don't access the image file in class, I suggest that they note down the slide numbers in their notes so that they can later match images to notes.

What I don't do is provide the image file well in advance. I am typically tinkering with the images until the last minute, and I explain this to the students when I describe what course materials I provide/don't provide and why.

And yet, despite providing quite a lot, one student wanted more. Not only did this student want me to provide the presentation file (with images) well in advance, but s/he also wanted me to print out the file each day and distribute a copy in class so that students could take notes directly on the images on paper.

OK, no problem. Just let me know how many images per page you would like. Should I print the pages in color? Don't worry about the cost. One-sided or double-sided? Of course I will collate the pages, but would you like them stapled or unstapled? If stapled, do you prefer the upper left corner or the upper right? Just let me know if you want me to punch holes in the margin so it will fit in a 3-ring binder. And just so no one feels bad about all the paper this will require, please rest assured that I will try to find paper made with a significant component of post-consumer waste.

And will there be anything else? Oh yes, OK, sure: I will highlight the points that will definitely be on the test. No problem.

This post is an example of how we focus on absurd little comments that are kind of (or very) negative in our teaching evaluations, even if the overall evaluations are positive. In the class I taught last term, 100%* would recommend me as an instructor! 100%* find me approachable and respectful! 100%* think I know my subject well and present it clearly! All of this makes me happy and I am grateful to my nice students who took the time to do the evaluations and express their satisfaction with the course.

Clearly, even the student who wished that I provided handouts in every class was overall happy with the class, so in this sense my colleague is wrong: students may ask for more, but it doesn't mean they are dissatisfied.

So, my typical approach to these situations is to consider all suggestions as constructive criticism, seriously ponder whether the suggestions are feasible, and use those that are and ignore those that are not. I may get ideas for how to explain my teaching philosophy in a different or more complete way (in this respect), even if I don't change what I do. To the extent that these evaluations by students are any use, for me, this is a good use for them.

* of the 89% of students in my class who did the evaluations

Evaluation Apathy

Perhaps the most dramatic result of the switch from teaching evaluations done on paper forms distributed in class to online forms accessible during a particular period of time at the end of an academic term has been the significant decline in student participation in the evaluations.

Some institutions/units/departments don't use online forms for this very reason, although online forms are certainly logistically preferable both during and after the evaluation process.

Using online forms in many cases reduces completion of the evaluations from > 80% (in-class forms) to < 40% (online forms).

I suppose you could argue that if students don't want to provide input, they shouldn't have to, but the concern is that teaching evaluations will lose what (debatable) value they have if < 40% of a class is providing comments. As long as teaching evaluations are used by administrators to judge teaching ability, the consequences for the instructor -- especially assistant professors and adjuncts -- can be dire if only those students who are unhappy are motivated to fill out the online forms (analogous to what happens to some professors on ratemyprofessors.com and similar sites).

From what I've seen, if a course has an online evaluation and the instructor does nothing to remind or encourage the students to fill out the online form, the participation is typically exceedingly low (<< 40%), even if there are reminders sent by whatever administrative office oversees the evaluation process.

There are things an instructor can do to increase participation, listed here in order of increasing evaluation completion rates (in my personal experience), from top to bottom of the list:

- make some announcements in class;

- make some announcements in class and send reminder e-mails;

- make some announcements in class, send reminder e-mails, and talk in class about why the evaluations are important (even for a senior professor) and why you care about getting their input -- you can even provide some examples of specific topics on which you would like input;

and the most effective method of the list:

- bribes and rewards.

Bribes work, but there are issues.

Some professors give extra credit if students do the online evaluations. According to some students I've talked to, some professors even say (or imply) that they can see which students fill out the evaluations and which do not, and promise extra credit only to those who complete the evaluations. In fact, correct me if I am wrong, but professors can't actually see individual names of who has and who has not filled out evaluations; we can see the number of completed evaluations and/or the % completion rate.

I think targeted bribes are unethical (i.e., saying that an individual student will get extra credit), but what about group bribes? Is it wrong to tell the entire class that they will all get a bit of extra credit if more than a certain (high) % of the class does the evaluations? Of course, giving the entire class extra credit is meaningless in some grading schemes, but let's ignore that for now and just consider whether the lure of extra credit for all is a good (and appropriate) way to encourage participation in teaching evaluations.

Pro: It takes time to fill out the evaluations, particularly if there are multiple instructors and TAs for each class. It's not a lot of time, but it's still a lot of clicking around to boring forms. Extra credit rewards a student's time and effort in an activity that the institution has deemed important. In this context, the extra credit is an incentive or a reward, not a bribe.

Con: Filling out an evaluation form isn't part of the academic work of the course, so extra credit for this activity isn't appropriate, even if it isn't a lot of extra credit. In general, it's not a good idea to bribe students to give their objective, thoughtful opinion about a course and its instructor.

Perhaps the best method is to offer a reward of some sort, but separate it from academic credit; e.g., the offering of prizes by some institutions (not instructors) to randomly selected students who complete evaluation forms. In the instructor-based scheme, every student in the class gets something (extra credit) if the target participation % is reached, whereas in the institution-based scheme, only some students get a reward. I don't think that is too discouraging to students, though -- low odds don't seem to discourage people from buying lottery tickets, for example.

If you use online evaluation forms, how do you (or your institution) encourage students to complete them?

Purposeless Pacing

From a teaching evaluation (not mine, but it could have been):

"The professor paced without purpose while teaching."

Discuss.

Questions for the purposes of data acquisition and discussion:

- Do you pace while teaching?
- Does your pacing have a purpose?
- If it has a purpose, is it a good purpose, a psychotic purpose, a pedagogical purpose, or what?
- Does anyone care? (i.e., Has a student ever mentioned it in your teaching evaluations?)
- Or, if you are a student: Do you care/not care if your professors pace, with or without purpose?

I confess: I pace while teaching. To the extent that my pacing has a purpose, it is so I can be a physical presence in various parts of the room at different times during the class, make eye contact with more students, listen to their questions better, try to see what they are seeing when I project something/write something at the front of a large classroom, or just because I get kind of hyped up when I teach and I feel like moving. I don't know if those are good purposes or bad purposes, but I think they add up to purposes, even if students don't know what they are.

As a pacer, I am therefore a fan of (my own) pacing. It didn't occur to me that it might also be distracting to students. I talk while I move around the room, so it's not as if I am walking back and forth silently, staring at the floor and thinking mysterious professorial thoughts during class. I can understand how that might be a bit disconcerting.

The questions above are intended to stimulate discussion of the important topic of professorial pacing. My discovery of this unexplored (by me) topic has demonstrated to me yet again that there is always something strange and new to think about, just when my energy and spirit have been nearly shattered, or at least flattened into a nanofilm, by committee work.

The Best of Times, The Worst of Times

As we enter the season of Teaching Evaluations for the term that, for many of us, is winding down (or up, depending on your mood), let us consider two issues:

(1) today's issue: Is the end of the term (but before the final exam) the (a) best, (b) worst, (c) as good as any other time to ask students to evaluate their professor's teaching?

(2) an issue for a not-too-distant future post (to give you time to think about it): What is the strangest comment you have seen in a teaching evaluation? (in your own evaluations or someone else's). I am particularly interested in strange comments that are not entirely off-topic.

A few months ago, a colleague was telling me about the results of a survey of undergraduates taking introductory Science courses at his institution. I was very surprised about the results of this survey (which I will not describe here). I was surprised because the alarming and depressing results were completely opposite to what I have observed (or thought I had observed) in my own classes and in my own department. So I asked this colleague for more information about the survey. How were the questions worded? When was the survey given to the students?

Aha. The answers to both questions explained a lot, and in particular, the one that is relevant to today's post re. the timing of the survey: It was given a few days before the final exam, when students are at their most stressed.

Yes, my colleague said, but if you give students a survey (or a teaching evaluation) after the final exam, most of them won't do it. And you can't give the survey/evaluation too early because the students won't have enough information about the class to respond authoritatively. And participation has to be voluntary, so you can't threaten them with consequences if they don't do the survey (and that might be counterproductive anyway). The only time to get a decent participation rate is just before the final exam.

OK.. but what if that skews the results? (and the people who are interpreting the survey results don't take that possibility into account and assume that there is a crisis because the students seem kind of stressed out?)

Assuming that student evaluations of teaching are going to continue to be employed by universities: If there were a way to ensure that (most) students would do the evaluation, is it "better" to have them do the evaluations just before the end of the term or after they get their grade? That is, is it better to have students do the evaluations when they aren't sure how they are doing in the class or when they know exactly how they did in the class?

And what is meant by "better" anyway? There are various ways to interpret that, but "better" in this context means a time when students will provide the most fair and thorough evaluations, after reflecting deeply on their own role in the learning process and how much they got out of the class.

I don't know the answer, but I do know that students are typically given teaching evaluations at a time when they are feeling a lot of stress about all the end-of-term activities (exams, papers). It would be interesting to know whether teaching evaluations and other surveys of student opinions of their courses would be substantially different after the term is over as compared to just-before-the-term-end. Surely someone has studied this?

Off Message

Here is a comment from my teaching evaluations for a medium-sized, mid-level course for majors in Science:

she is so kind and sweet :)

Well, that's.. special. Except that I wasn't trying for a "kind and sweet" kind of impression. OK, maybe "kind" -- I do try to be kind. But "kind and sweet"? No.

I don't mean to be ungrateful, but.. ick.

Sure, the alternative is worse: mean and bitter. But I have fangs and claws! I hiss when annoyed!

Last year I wrote about my surprise when a student hugged me after a final exam, to thank me for helping her (a lot) during the term. I worried that I was getting too "mom-like" and less professorial -- not in the stereotypical sense of being remote and detached, but in the awesome way of being authoritative and respected.

Can someone be "kind and sweet" and authoritative and respected?

The student who thinks I am "kind and sweet" wrote no other comments. Perhaps soon we professors will all have Facebook pages for our courses and, for our teaching evaluations, students can "Like" us (or not).

Professors With Boundaries

Of course I have heard (a lot) about that wondrous website, ratemyprofessors.com, but I have not been tempted to examine its contents. I think I looked at it briefly about 10 years ago when it first came into existence, but I didn't find it particularly interesting or illuminating, and did not return.

I looked at it recently, however, because I was semi-horrified to see a link appear on Facebook to the ratemyprofessors.com page of one of my colleague-friends. A student who is a "friend" of my colleague-friend had posted the link so that it showed up in the FB feeds of all the colleague-friend's friends. The student had done this to show his professor-friend how great the ratings were for a recent class my colleague taught, so the link was posted out of admiration and affection.

Even so, would I want a link to my evaluations on ratemyprofessors to appear on FB? Unlike my colleague-friend, I no longer friend (on FB) my undergraduate students, so this is unlikely to happen to me, but still I wondered: Would I want my (real) friends to see my teaching evaluations? Of course anyone can look these up at anytime, but I doubt most people would, unless a handy link appeared in their FB feed one day.

So I looked. My evaluations are quite nice; I need not lie awake at night in fear that a link will be posted on Facebook. The only surprise was how few there are, despite the large number of students I have taught in recent years. From the frequency with which ratemyprofessors.com is mentioned in articles and conversation, I thought that there would be more evaluations for someone who has taught as many students as I have.

I'm not sure what this means, but I have an unoriginal hypothesis, developed when I looked at the ratings for a colleague who had recently lamented the sorry state of his ratemyprofessor evaluations.

This colleague is an excellent teacher. The last time he taught a big intro class, the students clapped at the end of his last lecture. He got very positive (official) teaching evaluations. Nearly 100% of the students would take a class from him again. Students who hadn't expected to like the course instead found it interesting. The students loved him and were very happy with the class. Not one of them went to ratemyprofessors.com and wrote about this.

A few years ago, however, this same colleague was asked to team-teach the same course with a younger professor who was struggling with teaching. The course had some rough spots related to the times when the struggling professor was teaching, but my colleague worked hard to keep the course on track and to make it interesting (in terms of content) and fair (in terms of grades). The course was not a disaster, but it was not great. Quite a few students were inspired to go to ratemyprofessors.com and rant in a rather hostile way.

I can understand that students in this course may have had some difficulty separating the teaching abilities of the two different professors. To the students, it was one course, and they blamed both professors for the disorganization of one.

What is too bad is that the students in the later, excellent course were not similarly inspired to broadcast their happiness with their professor's outstanding teaching. If you only look at the evaluations for this professor on ratemyprofessors.com, you would not be tempted to take a course with this professor. That would be too bad, because he is an excellent teacher.

Of course, some professors do get rave reviews on ratemyprofessors.com. I was reading recently about the instructors who have the top ratings on this site, so there are some students somewhere who are sufficiently inspired by the excellence of their teachers to provide positive ratings.

At some universities, websites such as this may be the only way that students can get information (however flawed) about a particular professor. At other universities, the official teaching evaluations are available to students. I wonder if there is a difference in ratemyprofessors.com participation at different universities as a function of whether official teaching evaluations are available or not.

There are pros and cons of having official teaching evaluations made available to students, but at least these evaluations are a bit more comprehensive in terms of who participates. Or at least they used to be, back when evaluations were mostly done on paper in class, thereby capturing all those who weren't skipping class on the day of the evaluations. Now that many evaluations are done online, participation levels have dropped. Now that students can do evaluations anytime, anywhere, many don't.

The fear is that only those students who are unhappy -- perhaps the same ones who would be inspired to go to ratemyprofessors.com to share their complaints -- will now dominate the official (online) evaluations.

I personally have not experienced a downturn in evaluations, in part because I make a concerted effort to get students to do the online evaluations. I repeatedly remind the students (in class and by e-mail) about the evaluations, I request specific information in the evaluations, I explain why the evaluations are useful, and I share the %completion number with the class, showing the number increasing day by day. I typically end up with 80-100% participation, depending on class size (100% for small classes, closer to 80% for larger classes).

With these numbers, I can get a reasonably balanced view of how a course went, to the extent that one can get that sort of information from teaching evaluations.

It seems that we are all supposed to want to share everything about our lives with all our friends, but I guess I'm not quite on board with that yet. I am happy to share all sorts of information with my friends, but, however (mostly) positive my teaching evaluations are, I'd still rather not see a link to them appear one day on Facebook, wedged in between one friend's many graphic veterinary woes (TMI for dogs) and another's daily note to Jesus.

My evaluations might make more interesting reading for my friends than how I felt about my flight delays this spring owing to Icelandic volcanic ash (there are worse reasons for flight delays) or whether I liked the most recent books I read (The Possessed: Adventures with Russian Books and the People Who Read Them, by Elif Batuman, and yes, in fact, I loved it; also The Ask, by Sam Lipsyte.. both excellent books), but, if I have a choice in this particular case (and I may not), I'd still rather not share this aspect of my professional life with anyone other than my department chair and a few hundred students.

Tell Me Something I Don't Already Know

Like many of you who teach, I have recently been poring over my teaching evaluations from last term.

Brief attempt to forestall a few of the usual cynical comments: Yes, I care about teaching even though I am a full professor and my evaluations don't affect whether I get to keep my job. Yes, I read my teaching evaluations even though I know evaluations are flawed (except when they are really positive..).

Let's say that you feel pretty sure that your class(es) had overall gone well, and the teaching evaluations confirm that. So, if the evaluations just confirm what you already knew (or thought you knew), are they any use at all to someone like me who doesn't 'need' them?

Yes, in some ways. For example, in one of my classes last term I tried some new things: I tried a slightly new format and I introduced some new reading materials. Also, in this class the students do a lot of writing. I wanted to know what they thought about all of these things. I asked them in class and got some opinions that way, but not everyone expressed an opinion at the time. I had 100% participation in the teaching evaluations, so I got a lot more information that way. It was interesting to read that many of the students liked the writing assignments and the feedback they got on them.

And when I teach a new course -- either one that is new to me or one that I create -- it is good to get some feedback on the content and organization of the course.

Of course there are the usual vague comments: "The order of topics didn't seem quite right." End of comment. Which topics seemed out of order? What would be a better order? This information is not provided, but even vague comments like that can be semi-useful because they make you think about the organization and whether anything can be improved for next time.

After reading my teaching evaluations, I started thinking about the correspondence between professorial perception of whether a course went well and student evaluation of how the course went. Possibilities:

• Professor thought course went well : teaching evaluations agree
• Professor thought course did not go well : teaching evaluations are negative and hostile
• Professor thought course went well : teaching evaluations are negative and hostile
• Professor thought course did not go well : teaching evaluations are very positive

My hypothesis is that the last two are more rare than the first two.

I think we generally know when students are happy or unhappy with a course. We may be surprised at the magnitude of the general assessment -- e.g., you might be fairly sure that a course went well and then be pleasantly surprised at how positive the evaluations are; or you might be fairly sure that a course had some rough spots but be unpleasantly surprised and distressed at the level of hostility in the comments -- but I think we generally know the result to the first order.

Agree? Disagree?

Do They Know It When They See It?

This post continues with the theme of Teaching Evaluations.

As I've mentioned before, I write not only from the point of view of someone who is evaluated by students for my own classes but also as someone who, in the course of serving on various committees, has spent much time poring over other professor's teaching evaluations. In addition, I team-teach with various faculty, so I see them in action in the classroom once or twice a week for a term, and then compare notes with them later when we see our teaching evaluations.

Teaching evaluations are flawed in many ways and should be viewed carefully, but most students know a disorganized instructor when they see one. A large number of consistent negative comments about technical issues such as a professor's lack of organization during class, missed office hours, classes that go way over the scheduled time, assignments that are turned back very late (or never), and other logistical issues are surely accurate.

I doubt, however, that students can accurately evaluate a professor's level of knowledge, the depth of a professor's interest in being a good teacher and, in some cases, whether a professor means to be as rude as he/she seems.

I do not doubt that there are unambiguous cases of professors being rude and insulting to students. Professors who are deliberately rude to students should face negative consequences for this destructive behavior.

But consider these examples:

I team-teach with a professor who cares about teaching and who treats students with respect. He is not, however, a warm and nurturing person. His lectures are clear, he welcomes questions from students during class and answers all questions in a serious way, he conveys interesting and important information during lectures, and he gives exams and assignments that are fair and relevant to the material presented in class. Is he considered a good professor? Not really. Each term, there are comments in student evaluations that he is "unapproachable" and "arrogant". He gets comments about how just because he is so smart doesn't give him the right to look down on students. I can see how students would find him fierce in some ways, but I have been attending his lectures for years and have never seen him be rude to a student or discourage 'stupid' questions. And he works very well with undergraduate students in his research group. He hates reading his teaching evaluations at the end of each term because it is such a demoralizing experience. Why can't students see through the frowns and appreciate the atmosphere of learning and respect? To me, this is a prime example of flaws in the current system of student evaluation of teaching.

And here is an example of a student who interpreted insulting behavior where there was none: A failing student came to my office to talk about her grade. She was extremely upset, near tears, and convinced there was nothing she could do to get a passing grade. I showed her that her grade to date was within range of passing if she passed the final, or even if she showed improvement over her other exams, and I offered to go over the review questions and sample exam with her, question by question, in person or by email. She did not take me up on this offer, nor did her attendance in class improve. In my teaching evaluations, this student (who provided enough identifying and specific information in her comment so that she was the only possible person who could have written it) wrote that she was extremely insulted by how rude I was to her. Her evidence: When she was so upset about the possibility of failing the class, I smiled at her. My smile indicated that I thought it was funny that she was failing and that I was happy that she would fail. She was outraged about this. I should be fired etc. etc. I suspect that many professors have similar stories.

The difference between these two stories is that, in the first case, there is a consistent pattern of criticism. In the second, the student's negative comment was a hostile outlier in an otherwise positive batch of evaluations. Hostile outliers are painful to read, but they are easier to ignore than the other kind, which can look convincing if they persist over time.

I see many hostile outliers in other professors' teaching evaluations. Unless there are many negative reviews with specific comments, many of us who are professors who evaluate teaching performance, in part by examining teaching evaluations, don't consider the hostile ones as significant.

If there is a consistent pattern, we need more information, and we typically have more information -- in the form of peer evaluations, which tend to be quite comprehensive. Also, in cases where there is a clear lack of respect, there are typically many specific anecdotes. In the case of consistent comments along the lines of "He/she was intimidating" but nothing more specific, it's quite possible that the professor has not done anything overtly disrespectful to the students.

I am already anticipating my teaching evaluations for this term. Overall my classes went well (says me), but there was one student with whom I had a persistent communication problem that bordered on the bizarre at times. Another student in another class was sullen and angry the entire term; apparently not because of anything specific to me or the class, but who knows?

Do others anticipate negative comments from students after a particular incident or conversation? I wish I didn't, but it's hard not to.

Rage Against the System

Last week's discussion of teaching and teaching evaluations made me reflect more on my own experiences as a student and a professor.

As a student, I only had a few awful professors. When I ended up in a class with a professor who was a poor teacher, I was mostly disappointed rather than enraged about my wasted time or not getting my $'s worth or whatever. I was fortunate that most of these professors were merely inept teachers, not cruel or unfair. I had one cruel and unfair professor (during a year spent studying abroad), and that was more than enough.

I always got something out of a course, even if it was mostly from the reading. I recall one class that was dominated by neurotic students talking about themselves rather than focusing on the reading or general discussion topics. The professor seemed to encourage such behavior, and one day I couldn't take it anymore so I jumped out of the classroom window when the professor's back was turned (the classroom was on the ground floor) and escaped. Even so, I enjoyed the reading for that class and I got a lot out of being exposed to new literature and ideas.

As a professor, I see how hard my colleagues work at teaching (yes, even at a major research university ). And as a professor serving on committees that look at teaching evaluations for individuals over a multi-year period, I know that a common feature of the files of professors who start out as bad teachers is evidence for improvement, steady or dramatic (as discussed last week).

Outrage about uncaring and rude professors is justified, but I don't think that well-meaning but (initially) inept professors should be castigated in the same way. It's not as if every graduate student is encouraged to, or even given the opportunity to, develop teaching skills and somehow declines to do this. In most cases, the opportunities do not exist or are insufficient training for teaching a class as the major instructor. You can blame the system, but it's not fair to blame the graduate students who become the professors who have little teaching experience.

He Should Be Fired

'Tis the season for teaching evaluations (yet again, yet again), so in the spirit of this special time, I shall continue my intermittent evaluation of evaluations and teaching.

On quite a few occasions in my career, I have been a member of various sorts of evaluating committees that examine the research, teaching, and/or service activities of faculty within and beyond my department. In some cases these committees look at a single year of activity, and in some cases they consider a longer record. The latter cases can be quite fascinating, particularly with respect to an individual's teaching experience, because these multi-year records demonstrate in a dramatic way a very important but not surprising thing:

Most of us start out as not-great teachers.

I can't really defend my use of "most". Perhaps I should use "many", as I have not conducted a systematic study of this, but my impression from years of evaluating teaching evaluations is that "most" is probably correct.

To our students: We are sorry. Most of us don't mean to be lousy teachers when we first become professors.

But here's something cheering: Most of us figure out how to improve our teaching after just 1-2 classes within each level of course.

The worst evaluations I have seen for new professors have been for introductory level classes. Introductory level courses for non-majors at a university may be the most difficult kind of course to teach because of the (typically) large number of students, each with different priorities, interests, and learning styles, and because the physical environment (large lecture hall) may not be conducive to positive interaction between professor and student.

Some professors figure out how to improve their teaching with the aid of teaching development workshops; most figure it out just from experience. Student evaluations are important in the self-improvement effort because if something went really wrong in a course, many students comment on it. I always hope that among all the students writing comments along the lines of "You spent too much talking with your back to us while writing on the board", at least one, at some point during the term made a comment on this so that improvement could be made during the course. In many cases, logistical problems like that disappear from the teaching evaluation comments after 1 term.

In theory, some of us learn how to teach as graduate teaching assistants, but being a TA for a lab or a discussion section is very different from teaching a course, so, although TA experience does help, it's not enough. You have to teach a course to learn how to teach that course well.

So again, to the students: We are sorry, but unlike some other professions in which there is a training process for a major job component, many of us focus on research during our 'training' years and then are tossed in front of a class and expected to know how to deal with the complex logistics of teaching. You would think that, after spending many years sitting in classrooms, a person would know what to do and what not to do, but it doesn't work that way.

In some of my classes, students give presentations. These students are in the process of learning how to communicate clearly and effectively (in part by doing these presentations), but it always amazes me how common it is for them to do the very things that they no doubt hate in their professors: long text-filled slides (that they read), mumbling, lots of ummms, inability to answer questions on the spot, no (obvious) statement of the main point, or poor organization of complex information. Clearly, most of us need practice to learn how to convey information in an effective way to a particular audience.

And, unfortunately, we have a system in which professors practice on a class or two before eliminating some problems.

Years ago, I had a minor surgical procedure during which I noticed that the doctor was becoming agitated. He was also taking a very long time. I started getting nervous so I asked him if everything was going OK. He admitted to me then and there, while holding a knife that had already been used on me, that he had never done this procedure before by himself and it wasn't going as well as he had hoped. I asked, as calmly as I could, if it would be possible for him to get another doctor to help him, and he agreed that this was a good idea. A more senior doctor was sought and finished the procedure. I wasn't too happy about being the young doctor's first solo-surgery experience, but I figured: someone has to be the first patient.

I bet that now, years later, that doctor calmly does these procedures often and well. At least I hope that is the case. It is the case if the doctor analogy is at all applicable to professors whose first solo-teaching experience does not go very well. (The analogy does break down a bit after that, though, so go ahead and send your bitter anti-tenure rants and your "I had a professor who had been teaching for 47 years and was horrible" rants if you really must.)

I chose the description "not-so-great" above because, from what I've seen and read, most of the teaching problems are not course-destroyers. That is, most of us are not total disasters when we first start teaching as professors, and even among those who are, in all but rare cases there is improvement after a term or two.

I empathize with the students who are being practiced upon and who may have emotional scars (much like my little surgical scar that could have been avoided with a more experienced doctor), and I am always impressed by those who write evaluations such as "I know this is your first year teaching and [insert helpful suggestions]."

I even feel sorry for those who feel compelled to write, typically in all-caps: PROFESSOR X SHOULD BE FIRED. These students clearly spent months feeling angry, frustrated, and perhaps afraid of the effects of this awful course on their academic careers. I feel sorry for them, but I wish they could know that the professor needs advice (from both students and other professors) and experience, not firing. Advocating firing a new professor who is learning how to teach is not a very constructive suggestion, although perhaps in extreme cases such dire comments help signal the severity of the problems that need correcting.

That said, if someone has been teaching for many years and/or clearly doesn't care about teaching well and/or or has failed to learn how to communicate effectively and/or does not know how to deal well with student questions and course logistics, then such drastic comments are fair.

But in a professor's first year? No. Even though I appreciate how difficult it is for students who are in that first class of a new professor who is struggling to learn how to teach, I think we should all try to figure out a system that would minimize the painful disasters, try to be patient with annoying-but-not-too-terrible problems, and find constructive ways to make the teaching/learning experience better for faculty and students.

Except One (2009 edition)

Last June I wrote about how every student but one in my medium-sized course for science majors gave me a positive evaluation for my teaching. What I wrote last year applies to this year as well for another medium-sized course for science majors that I taught this past term. Last year I wrote:

In the evaluations for my class this time, every student except one said that they learned a lot. Every student except one said they would recommend my class to others. Every student except one said that they would recommend me as an instructor. Every student except one said that I treated students with respect, was approachable, gave timely feedback, was organized, and so on.

But here's the difference between last year and this. Last year I wrote:

I know which student hated the class. This student was unremittingly rude throughout the entire academic year.

and I asked the question:

Is it better to know in advance that at least one student hates your class, or is it better if you don't know until you read your evaluations?

I now have more data to answer this question. This year, I have no idea who the exception is. I was shocked that one student hated me and the class so much and I was completely unaware of this until I read my evaluations.

I would understand it more if this were my big intro course with hundreds of students. In a big intro course, it's not unusual to have a few students who check off all the most negative ratings on an evaluation. In my big class this past term, I was surprised that I only got one extremely negative comment (maybe one of the cheaters I busted? the student I caught lying about needing to miss a quiz for an athletic event? someone else entirely?) and one moderately negative evaluation (from someone who hated the fact that when I was asked a question by email, I typically explained how and where to find the information rather than just giving the answer directly). I expect those sorts of evaluations in a big class, and as long as there are only a few of them, I don't worry about it.

In a small to medium-sized course where you know all the students by name and see them around the department all the time and chat in the halls etc., it's troubling to find out that one of them was extremely unhappy throughout the course and never said a word despite having ample opportunity. The one unhappy student wrote no specific comments to explain what exactly he or she hated about me and the course, but checked off all the most negative ratings on the multiple choice part of the form.

So, is it better to know in advance that at least one student hates your class, or is it better if you don't know until you read your evaluations?

Both are lousy situations, of course, but I guess overall I'd have to say that I prefer the latter. I feel great regret that I didn't know one of my students was so unhappy with the course and there was no time to discuss the problem, but I enjoyed teaching this class this term, thinking (delusionally, as it turns out) that all of the students were also enjoying the class.

Last year, the one extremely hostile and vocal student had a serious negative effect on the the class and my feelings about teaching it. Each day of the course last year, I dreaded going into the classroom and seeing that student sitting there, arms crossed and frowning, in the back of the room, preparing to whine about something. The rest of the class was intimidated by this student and the atmosphere of the entire course was affected.

This year, the class seemed cheerful and filled with motivated, perky students who were very interactive about asking and answering questions during the class. It was a pleasure to teach that group of students.

So I lost one student along the way and I am sorry about that, but if you forced me to choose between the two unappealing options, I would choose the unknown hostile student over the in-my-face hostile student.

Your Post-Evaluation Persona

A conversation with a colleague today about Teaching Evaluations made me wonder if I had scratched the dark underbelly of college-level teaching or whether I was just desperate to do another poll on a strange topic. Or both. As you may have guessed, I have a question for my readers who teach:

Do you change anything about your teaching behavior in the few classes that remain after teaching evaluations have been completed for your class?

Or, at the very least, do you feel a sense of relief after evaluations are done because you know that you can make innocent mistakes and not pay the same penalty for them? In fact, the topic came up when my colleague said that he didn't want to do online evaluations because the online evaluation period lasted too long and didn't give him those last few post-evaluation classes in which he could relax more as he taught.

After an admittedly shockingly brief investigation, I have determined that in the apres-evaluation near-end-of-semester time, some of my colleagues don't feel quite the same compulsion to answer annoying questions like "What is going to be on the test?" or "Are you going to ask anything from the classes in the week before Spring Break?" My colleagues report that they answer these questions, but perhaps not as nicely or completely as before the evaluations are done. Smiling may become more optional when answering such questions apres-evaluation. Having teaching evaluations completed can be rather freeing, but not in any sinister way.

What this question really gets at is the extent to which our behavior is governed by the looming prospect of teaching evaluations. Does the fact of teaching evaluations make (some of us) nicer? Is that a good thing?

I am not dealing with the issue of whether teaching evaluations make us better teachers by giving us constructive feedback or biting criticism, albeit too late to change anything to help a particular class (e.g. "I would have gotten more out of the class if you didn't just read endless text slides in a monotone every single class"; "You suck", etc.).

No, I am asking whether the prospect of teaching evaluations affects professorial behavior when interacting with students. Hence the question: Are you more patient, kind, and/or polite than you might otherwise be because you don't want to be slain in your evaluations for being cranky and terse with students, even if you are being completely insincere? And: Does your niceness level decrease, even if ever so slightly, after evaluations are done?

The Perfect Storm/Classroom Edition

Not long ago, the department chair asked me to look at the teaching evaluations and grades for a class taught last year by a less experienced (and non-tenure-track) professor. The grades were similar to ones that colleagues and I had given to this same group of students in previous classes. The evaluations, however, were a bit on the unkind side, to put it mildly, and the chair was concerned.

When I read the evaluations, my heart sank. Then I kicked myself for not being more assertive about helping my younger colleague. From time to time during the semester, I asked her how things were going with her class, and she always said everything was fine. I should know by now that just asking that question isn’t the best way to be helpful, and it puts a junior colleague in an awkward position.

Our department has no system for peer evaluation of teaching, but it should. If there is a system of peer evaluation for everyone who teaches, there is no stigma attached to having other faculty evaluate your teaching. My previous university had peer evaluation of teaching. Sometimes it helps, and sometimes it doesn't, but surely it is better than not doing anything until it is too late for a particular class? I don’t know if I could have helped this colleague by sitting in on some of the classes, but she made some basic mistakes* that might have been avoided with better mentoring.

* Examples: moving the date of an exam without sufficient notice and/or consultation of students; relying too much on PowerPoint presentations; not dealing well with questions from stressed out (rude, angry) students.

I can appreciate how frustrating it must be for students to be in a class taught by an inexperienced instructor, particularly if the class involves difficult concepts. Even so, I think that some of the students in this class were especially unforgiving. It was a challenging class, and this group of students had struggled in classes that were less challenging. When faced with a more difficult course and a somewhat (but not completely) inexperienced instructor, conditions were set for a really bad experience for all concerned. A subset of the students were particularly vicious in their evaluation of the course.

Based on what I’ve seen over the years, requirements for this particular type of extreme negative reaction seem to be (1) a challenging class taught by (a) an inexperienced instructor who (b) does not project a confident image and may (c) lack authority (e.g., non-tenure track faculty, adjuncts, grad students); and (2) a group of students who develop an unforgiving and increasingly negative attitude about every imperfection in the class. The collision of a group of unhappy, confused students with an inexperienced instructor who is vulnerable owing to a lack of status (adjunct or visiting professor) and/or lack of confidence, creates a perfect storm of negativity and an unpleasant learning/teaching experience for all.

Although I think that some of the students could have handled the situation more constructively during the course, this is not a ‘blame the students’ rant. The department should have done more to help the inexperienced instructor during the course, and I should have done more as the unofficial mentor of this instructor. You can’t infuse someone with decades of teaching experience instantly, but it should be possible to provide logistical or psychological (confidence-building) support to help a course while it is in progress.

Once a bad teacher, always a bad teacher? Certainly not. Some people are naturally great teachers, but many struggle in their first few years of teaching. The challenge is to keep your confidence from being further undermined, as that can create a spiral of disaster experiences from which it becomes ever more difficult to recover. If you stay calm and assess what went wrong and what went right, you can fix the problems and get on track for having a more positive teaching experience in the future.

Evaluating Teaching Evaluations

Teaching evaluations are best viewed in a general, cosmic, slightly unfocused way as an indicator of some aspects of a teacher's methods and effectiveness. I just got my spring evaluations yesterday, and I am pleased with the results, but that won't stop me from musing about the bad (and good) aspects of evaluations.

I think that if you look at someone's teaching evaluations for a range of classes (small and large, majors and non-majors, graduate and undergraduate) over 3-5 years, you will get a sense for whether that person is a 'good' teacher or not. Ideally, 'good' means interesting, caring, challenging, accessible, organized, and fair, and doesn't correlate with a propensity to give A's for mediocre work and effort.

At my previous university, I had 2 teaching mentors who sat in on my classes at random times during the semester and provided me with feedback. They were supposed to tell me in advance when they would be sitting in, but they didn't. This was fine with me because it meant they saw the class more as it was on a routine day, rather than on a day when I had prepared in a different way knowing I'd be monitored. I didn't really get any useful feedback from these teaching mentors except about some technical issues on how to more effectively use the A/V equipment in the classroom. Even so, I liked the fact that there was an additional evaluation method of my teaching other than just the student evaluations.

At my present university, our only feedback comes from the teaching evaluations that students fill out at the end of each semester, so these evaluations are the sole evidence for whether we are good teachers or not. That's why it's important that the evaluation process be improved.

Problems with the current process (written from the perspective of someone teaching at a large university):

1. Some of the questions are strange, irrelevant, offensive: I have written about this before, but some of the primary questions that are the basis for the quantitative aspect of the evaluations are inappropriate, e.g.,

- A question about the physical environment of the classroom asks students to rate me on something over which I have no control.

- A question about how the students would rate the professor's knowledge of the course material is offensive as written. That question could be worded to ask how effective the professor is at conveying their knowledge to students, rather than asking the student how much they think the professor knows.

- A question about how much a student learned in the course is not only rating the effectiveness of the teacher, but also how much effort the student put into the course. Sometimes the two are related (students may put more effort into an interesting and well-taught class), but sometimes they are not. A student who does not attend class regularly and doesn't do assignments etc. will not learn much in the course, yet he/she gets to rate how much they learned in the course as part of the evaluation of the professor's teaching. In theory, these students are a small minority and their low ratings will be outliers in an overall positive evaluation, but their effect on the evaluation depends on class size.

The questions I think are good and useful include:

- A very general question on the professor's overall teaching ability.

- A question or questions about the professor's level of respect, concern, accessibility for students.

- A question about whether graded work is returned in a timely way.

Even for those questions, a student who is not doing well in a course will in some cases give a professor a low rating, but again, if the class size is sufficiently large, these will be outliers.

And then there are the detailed comments. Reading these can be alternately heart-warming and enraging. I am fortunate that this semester I got the heart-warming kind. More typically there are at least a few that will say "I wish you had done X", where X is something unreasonable (e.g., "I wish you could have given us all the answers to the quiz questions in advance so that we could have studied them.").

And of course there is no chance for rebuttal/response, and there is no way to fix any problems for that specific class. That can be frustrating.

In summary: it's a flawed process that sort of works in a general way.

Evaluated

Today I got my course evaluations for last semester, just in time to help me prepare for the next semester.

I was relieved to see that my minimalist syllabus (with extra info on the web) from last semester was viewed favorably: 100% of the class of ~ 30 students said the syllabus was accurate and gave them the information they needed.

- 100% of the students felt the course stimulated them to think critically about the course material, said they would take another course from me, and believed that I had 'exceptional' knowledge of the course material. Thank you!

- 97% said I am prompt in returning graded material and providing feedback about student performance. The one student who disagreed with this statement must have been very cranky -- I ALWAYS give exams and homework back in the very next class after the exam or homework due date. Or maybe waiting 2 days seems like a long time to this person?

The most variable score was for a question related to whether the amount learned in the course was similar or different to student expectations. I never know what to make of this question. If a student expected the course to be bad and it was, then the course met their expectations and I would get a high score, and a high score is generally considered good for this question. If a student expected the course to be bad and it wasn't, I'd get a low score. The result is usually an uninterpretable range. Last term, most students seem to have learned more than they expected. I suppose that's good, but it seems to suggest they had low expectations, and that's not so good.

At my previous university, faculty had no choice about teaching evaluations being published for all to see, and I had no problem with this, although I would have preferred it if the reviews were accompanied by some factual information about the courses in addition to the student evaluations.

At my current university, we have to give permission for certain parts of the evaluation to be released. I haven't done this, in part out of inertia (you have to take the initiative to do it, and I just haven't), and in part because I think the evaluation questions are very poorly worded at this university. The question about whether the course met expectations is just one example.

Another very prominent question on the evaluation form has to do with whether the student liked the classroom. I don't get to choose my classroom, so what does this have to do with me or my teaching abilities? This question is always my lowest score. Last semester, 12% really liked the classroom, 69% had ratings ranging from good to very good, 15% thought the room was just OK, and 4% thought it was very poor. If my evaluations were published, I trust that students could sort this question out from the ones that have real meaning for a course, but even so.. it bothers me to be rated on this.

I suppose this question could be used to identify major problems with a particular room, but there must be other ways to acquire that information. The main value of this how-did-you-like-your-classroom question, however, is that it is entertaining for faculty to compare their results for the very same room. The 'goodness' of any particular classroom tends to fluctuate with time of day (early in the morning, the room is not good; it gets a little better up until lunch; gets worse again just after lunch, and then gets better until late afternoon). It is also interesting to compare the Room Question results for faculty who team-teach the same course in a single term. The results can be remarkably different. This shows that evaluations are best used to get a cosmic sense for whether a course worked well or not, and the results shouldn't be picked apart in detail.