Monday, May 02, 2011

Evaluation Apathy

Perhaps the most dramatic result of the switch from teaching evaluations done on paper forms distributed in class to online forms accessible during a particular period of time at the end of an academic term has been the significant decline in student participation in the evaluations.

Some institutions/units/departments don't use online forms for this very reason, although online forms are certainly logistically preferable both during and after the evaluation process.

Using online forms in many cases reduces completion of the evaluations from > 80% (in-class forms) to < 40% (online forms).

I suppose you could argue that if students don't want to provide input, they shouldn't have to, but the concern is that teaching evaluations will lose what (debatable) value they have if < 40% of a class is providing comments. As long as teaching evaluations are used by administrators to judge teaching ability, the consequences for the instructor -- especially assistant professors and adjuncts -- can be dire if only those students who are unhappy are motivated to fill out the online forms (analogous to what happens to some professors on ratemyprofessors.com and similar sites).

From what I've seen, if a course has an online evaluation and the instructor does nothing to remind or encourage the students to fill out the online form, the participation is typically exceedingly low (<< 40%), even if there are reminders sent by whatever administrative office oversees the evaluation process.

There are things an instructor can do to increase participation, listed here in order of increasing evaluation completion rates (in my personal experience), from top to bottom of the list:

- make some announcements in class;

- make some announcements in class and send reminder e-mails;

- make some announcements in class, send reminder e-mails, and talk in class about why the evaluations are important (even for a senior professor) and why you care about getting their input -- you can even provide some examples of specific topics on which you would like input;

and the most effective method of the list:

- bribes and rewards.

Bribes work, but there are issues.

Some professors give extra credit if students do the online evaluations. According to some students I've talked to, some professors even say (or imply) that they can see which students fill out the evaluations and which do not, and promise extra credit only to those who complete the evaluations. In fact, correct me if I am wrong, but professors can't actually see individual names of who has and who has not filled out evaluations; we can see the number of completed evaluations and/or the % completion rate.

I think targeted bribes are unethical (i.e., saying that an individual student will get extra credit), but what about group bribes? Is it wrong to tell the entire class that they will all get a bit of extra credit if more than a certain (high) % of the class does the evaluations? Of course, giving the entire class extra credit is meaningless in some grading schemes, but let's ignore that for now and just consider whether the lure of extra credit for all is a good (and appropriate) way to encourage participation in teaching evaluations.

Pro: It takes time to fill out the evaluations, particularly if there are multiple instructors and TAs for each class. It's not a lot of time, but it's still a lot of clicking around to boring forms. Extra credit rewards a student's time and effort in an activity that the institution has deemed important. In this context, the extra credit is an incentive or a reward, not a bribe.

Con: Filling out an evaluation form isn't part of the academic work of the course, so extra credit for this activity isn't appropriate, even if it isn't a lot of extra credit. In general, it's not a good idea to bribe students to give their objective, thoughtful opinion about a course and its instructor.

Perhaps the best method is to offer a reward of some sort, but separate it from academic credit; e.g., the offering of prizes by some institutions (not instructors) to randomly selected students who complete evaluation forms. In the instructor-based scheme, every student in the class gets something (extra credit) if the target participation % is reached, whereas in the institution-based scheme, only some students get a reward. I don't think that is too discouraging to students, though -- low odds don't seem to discourage people from buying lottery tickets, for example.

If you use online evaluation forms, how do you (or your institution) encourage students to complete them?

33 comments:

Anonymous said...

At my current institution, you can't access your final grade for a class unless you've completed both a course evaluation and an instructor evaluation for that class.

Tam said...

One of my professors requires all students to do the evaluation. After you do it, when you log in, it shows "completed" for that course, so we have to print that out and bring it to him. If he doesn't receive it by the deadline he claims he takes 10% off your course grade every day.

I don't know if he actually follows through on this threat, but I observe that he got 91% participation from his undergrads last semester.

(I'm a grad student, and our evaluation data isn't posted on the online system.)

Anonymous said...

At my undergrad institution, we used online evaluation forms that were consistently filled out by the vast majority of students in the class.

In order to fill out an evaluation, you need to log on to the university's website with your ID. Students who do not fill out evaluations for all of their classes for a given term are not allowed access to any class evaluations for any previous term (which are visible to everyone at the university). Access to previous class evaluations are incredibly useful in choosing which courses to take or which professors to avoid, and thus almost all students dutifully filled out their evaluations each term.

The system worked well.

Anonymous said...

As a student I chronically forget to fill out online evaluations, though I tend to give more (sometimes paralyzingly time-consuming) thought than I think most students do when faced with paper evaluations in class. I think requiring completion of evaluations to view final grades is a very sensible situation and would force me to fill out evaluations, which I would often like to do but often repeatedly slips off the radar especially during finals.

Most recently, as a grad student, the most effective cue was solidarity - I made sure to fill out my evaluation for the TA who reminded us that apparently to get credit for TAing a course you need at least a 50% evaluation response rate, and once it was on my mind I filled out the course and faculty evaluations as well.

Anonymous said...

My institution rewards students who complete the evaluations by making their final end-of-semester grades (for all courses) viewable about five days earlier than for students who fail to complete one or more course evaluation. We've been using electronic evaluations for about three years, and from the get-go the response rate has been almost exactly the same as for the paper forms.

Hal said...

My institution recently moved to the online format and my unit has reserved computer labs for instructors who would like to take a walk with their class. As a result we had one of the highest participation rates in the system. Our size is such that this works but it's probably not a general solution.

I believe the burden of achieving good participation needs to fall on the institution, or at least the instructor should not be held responsible for poor participation.

I value student feedback but data being collected via the formal student evaluation is intended to be used by the institution. I typically do my own online evaluation midsemester and could do the same thing at the end. In fact, this would enable me to collect data more relevant to my courses than the standard forms.

Lindsay said...

In one of my undergraduate universities we had the paper evaluations. Even though many of the students filled them out, I would see them fly through them, eager to leave the classroom early, with no possible way that they were actually reading the questions. As a graduate student now, my university has all evaluations online. It is distributed completely by the department secretary. Many of the freshman labs that I teach offer 2pt extra credit (which is very little of their grade). The secretary is able to email the names of all participants. I have seen high (~80%) participation rates even for courses that do not offer the extra credit. I like the time that the online systems gives me. I can take the evals when I want (within a window) and take as long as I want.

Wendy said...

I bring donuts to the final is the response rate tops 80% by the deadline (last Friday of classes; before finals week).

macmaxbh said...

I assume the real reason for the high response rate on paper evaluations is that teachers would give in-class time to do them.

It's harder with online evaluations, but saying beforehand "I'll give you time at the end of class to do your evaluation, bring your laptop" and then giving the end of class over to the evaluations (letting students who have already done them or don't have a laptop go, encouraging those without laptops to go to a nearby computer lab) would in my mind improve the on-line rates.

I don't think there's anything intrinsic about online evaluations that would cause them to have lower response rates (the opposite, in fact).

Maarja said...

At our university, all registration for courses is done online (in the same system where evaluation forms are filled out) and you cannot register to next semester's courses until you have completed all the evaluation forms. Pretty effective... :)

Prof-like Substance said...

My institution tried to hold back grades until students filled out the online form, but there was still not very high compliance and the university had to relent on it's policy because it made such a mess of registration for the following semester.

And now we're back to paper. Ugh.

studyzone said...

The university I taught at this past year recently switched to an all-online evaluation process. If students do not complete the evaluation by the official end-of-semester date, their transcripts are withheld and they cannot view their final grade for any class. While I know that the university is getting >80% response rate due to the change in policy, I wasn't there long enough to assess whether the switch to mandatory evaluations has had any effect on student responses.

Female Post-doc said...

I think having it rolled into grade access as Anon @ 12:00am suggested is a good way for the University to show they think these are important and takes the onus off the professor.

That said, those non-university online sites don't have reviews for only bad professors. They also have good reviews for instructors generally perceived to be attractive/hot/sexy by the students. :-)

Eileen said...

My uni's system is the same as Anon 12:10--no access to final grades until the eval for that course is finished. I think if I were a student, it would feel a little heavy handed, but it gets high participation rates.

Sophia said...

Promise them free pizza, always seems to motivate students.

Anonymous said...

The following would probably work:

Pizza/Doughnuts (for a <20 student class) or candy (for a giant class) during the final but only if 75% or more complete evaluations. With a reminder of what % have done it each of the last 4 class periods.

Anonymous said...

I have my students do online evaluations in class.

I can also see who has completed an evaluation and who hasn't after a certain threshold of evaluations have been done (I think 5).

Anonymous said...

We get >80% participation in online evaluations by allowing early access to grades for students who fill out the form. We don't withhold grades for students who don't fill out the forms - they just have to wait to receive them by snail-mail.

Anonymous said...

Similar to the first Anonymous, my current institution does not allow students to access grades until all evaluations are completed. The university sends a reminder email about the deadline for completing evaluations as well.

Lauren said...

During my undergrad, I had a per-course instructor (non-faculty) make it a competition between the 2 or 3 courses she was teaching. The group with the highest turnout got homemade cookies at the final exam. I thought this was reasonable and ethical motivation.

Anonymous said...

I will make cookies or brownies or some treat for everyone on the last day of class if more than 80% of the class has filled out the evaluation.

Female Science Professor said...

I don't think students should be punished for not doing an evaluation, but a positive incentive is fine. There are lots of good ideas here.

Anonymous said...

At my current institution, students who do not fill out the online evaluation receive an incomplete in the course.

I have no idea how well this works for honest evals (I'm a postdoc). But it seems coerced and harsh.

Anonymous said...

That is way too harsh. Is there a student organization that has tried to get that overturned, or is this system OK with the students?

Anonymous said...

At my undergrad institution you couldn't get your grades until you filled out all your evaluations. They gave you a Charms lollypop when you finished evaluations.

My current institution is about to make all evaluations online (before we had a choice). I dread this.

Science Undergrad said...

I disagree with the idea of any kind of incentive or compulsion in the matter of evaluation forms. Knowing college students, many will fill them in without thinking about it very much in order to get to the end result. It skews the data in a study already unfortunately prone to bias.

I think that the students you want giving you feedback are the ones who care enough to give it a little thought without having to be bribed.

Reminders are awesome, though. I always intend to do my course evaluations, but I sometimes forget the due date is coming up... close to finals as they are.

Anonymous said...

At my current institution, instructors can see how many students have filled out the evaluations (but not which, or what their responses are). To boost participation I tell my students that I'll put an extra-credit problem on the final, and how many points it will be worth depends on what percent of the class fills out evaluations by the end of the time they're available.

Anonymous said...

I think evaluations are a waste of time and never bother to remind the class to do them. Students do get email reminders every few days from the university until they've done it.

EngineeringProf said...

My university offers no incentive or punishment and in fact students can still complete the evalauations online after taking the final exam. This leads to low returns and lower than expected reports for profs that give hard exams.

Anonymous said...

I bribe my students. I promise them bonus points on the last HW if the % of completed course evaluations tops 80%. The bribe has a much larger psychological effect than numerical influence on their grade. I don't really worry that these minor additional points will somehow skew my grading.

Nicole said...

At my undergrad institution a common bribe to fill out your online evaluations was for professors to agree to drop your lowest homework grade. Somehow they could see who had and who hadn't filled out the evaluations. (Or maybe they were just lying!)

It was a nice thing to do and I don't think it actually skewed people's grades too much. Although it did result in me being able to just skip turning in a homework set altogether once!

Anonymous said...

In my institution students also have to put in evaluations in order to see their grades, BUT... they can "opt out" of evaluating any course. So the requirement is basically to go into the online system, and go course by course.. but then you can opt out of all of them. At least it means that they can't just forget to do it. But they are not coerced into actually giving feedback if they don't want to.

Anonymous said...

I'm an undergrad at a mid-sized university. Departmental evaluations may still be conducted in class on paper. But within the past year, the school evaluations (Engineering, Biz or Arts & Sciences) are done only online.

I'm assuming that participation is high for two reasons:
- The thing will NOT stop sending evaluation reminders. Every two days you don't complete the evaluation, it sends you another urgent e-mail. The only way to stop it is to complete the evaluation.
- Your final grades are delayed by a week or two if you don't complete the evaluation.

I'm not privy to the exact response rates, but I presume them to be quite high.