Room for Improvement

Student comments on my teaching of a particular course:

Great professor!
I have enjoyed this class!
I liked the readings.
This course required too much previous knowledge.
Professor very helpful with homework.
Homework very useful for class.
Well-constructed lectures.
Very organized lectures.
She speaks very clearly.
She answered my homework questions.
She provided images and charts to supplement the subject matter.
The in-class exercises were helpful.
I liked the practice exercises we did in groups during lecture.
I liked that she asked questions during class and this helped deepen my understanding of concepts.
Useful supplementary material to help us understand lecture material.
She explained the topics completely in class. Didn't use a textbook as a crutch.
It was great that lecture and lab material were well coordinated.
She was always ready to answer questions.
She was always willing to help with any questions.
She provided the subject matter very clearly.
The last project was too much work for this level of class.
Lecture presentations very clear.
I liked the in-class exercises.
You should improve your teaching methods.

Note that almost all of the comments are in the 3rd person (except for the last one), as if the students were writing to someone else about me, rather than writing to me with feedback. I don't know if it matters in terms of type and level of feedback whether the student is imaging an unknown audience or speaking directly to me (?). At evaluation time, I give a little talk to the class about the importance of this feedback and how it is used by instructors and the department/college/university, but I think there is still general confusion among students about what exactly the purpose of these evaluations is and who reads them and whether anyone cares what they think.

These are overall nice comments, and unfortunately also rather classic in that the criticisms are too vague to help me understand what the specific complaints are.

The last comment, despite being too vague to be useful in any specific way, is absolutely right. Despite being deep into my mid-career years, I don't want my teaching to fossilize. I want to improve. In recent years I have attended teaching workshops and gotten some ideas from those. When I team-teach, a faculty colleague is in the classroom with me, so I get some peer feedback. And last term, I jettisoned the too-long and too-detailed textbook and provided focused readings, including some that I wrote myself. That seems to have worked quite well (or at least no one said they missed having a textbook), so perhaps that counts as an improvement. I would also like to do some new things involving e-learning and have been to some workshops and meetings about that.

I am thinking about teaching because I was just looking at my evaluations, though mostly I am enjoying having lots of uninterrupted time for research. This week I even managed to submit a manuscript on which I am primary author. It's been about two years since I've been able to do that (and I don't mean to imply that I did it alone -- an excellent colleague was essential to the completion of this paper).

As I was finishing the paper (and a related grant proposal) recently, it occurred to me that I could create a new teaching module based on this work and incorporate it into the class for which I just received teaching evaluations (not, of course, as extra work but replacing some older material). Probably more than any major change in teaching style, a realistic way that I can improve my teaching is to find good ways to incorporate new material -- specifically, integrating New Science with Classic Science, so that students learn the fundamental stuff without which they are incomplete as scientists and people and yet are also exposed to new things that help them see where the field is at (including being exposed to unresolved questions that might inspire them).

Anyway, it's been a busy summer so far. My father recently asked me if my husband "also has the summer off" and I was actually quite calm about it this time. Have you had a similar conversation with anyone yet this summer? Parents? Neighbors? Friends? Students? Assuming that you do in fact work in the summer even if you are not teaching, did you (1) smile serenely and let them continue to exist in ignorance; (2) correct them (a) calmly, (b) not calmly; or (3) lapse into stony silence (if having a conversation) or send a glaring emoticon (if in e-contact)? (or other..).

Liveblogging the Exam

Although taking an exam is most certainly more stressful than giving an exam, giving an exam can be quite stressful. I am not asking for sympathy, I am just stating a fact.

The most stressful exams to administer are those in large classes in which students are packed into every available seat and there may be (alas, too often) issues with cheating. You can devote considerable time and effort to anti-cheating activities such as giving multiple versions of exams, you can have students sign an honor statement, you can patrol classroom non-stop during the exam, or you can just hope for the best.

Giving an exam to a small or medium-sized class is less stressful because the logistics are easier, but that is not to say that giving an exam even in these circumstances is lacking in stress. Or, at least, that is my opinion. Is there anyone who would rather give an exam than have a regular class? I would much rather have non-exam class time. [I am deliberately not addressing the possibility of not having exams at all. In some courses I do not give exams, in others I do, depending on the course.]

OK, so I am about to give an exam in a medium-sized class. I am not dexterous enough to blog while handing out the exam (and the TA does not seem to be in evidence), but I can combine semi-live and liveblogging to try to capture the essence of the experience from the front of the room.

I enter the room. They are all here, in their seats, staring at their notes. This has been a very punctual class so I am not surprised they are all here on time. In a more typical class, students would appear throughout the first 10+ minutes of the class, even on an exam day, making a lot of noise as they rush in, grab an exam, find a seat, deal with the logistics of finding a writing utensil and putting their water bottle in a suitable place

Put your notes away! This takes a moment. I try not to let it eat into exam time (there is another class in this room right after ours), so I start handing out the exam to the nearest row of students who are note-free.

Can we start now? I should remember to say that they can start as soon as they get the exam but sometimes I forget and then someone always asks. It is not a large class, so the time difference between those who get the exam paper first and those who get it last is about a minute. If anyone in the back needs that extra minute, they can have it at the end. In a large class, I need a fleet of assistants to help hand out exam forms so that no student has to wait too long to get the exam or to get their question answered if they have one during the exam.

The room is never totally quiet. Someone is always turning a page, even if the exam has.. one page. And certainly if the exam paper has 2 pages: rustle rustle rustle. The second-most common noise is erasing. There is a lot of erasing going on at this very moment. Some students have very impressive erasers.

Two students just had questions about different exam questions. Both were answered easily by my pointing to a key word or words in the exam question. In both cases the student immediately saw that the answer to their question was right in front of them and thanked me.

The first student is done with the exam, halfway through the allotted time.

The second student just finished, with 20 minutes left to go. Make that three students.

Are the ones who finish very early the ones who are doing well in the course so far? In fact, there does not seem to be a strong correlation between those who are doing very well or very not-well in the course (to date) and those who finish the exam early, just as I predict that there will be a random collection of students (with respect to course performance) at the very last second of the exam.

At various times in my teaching past, I have timed myself taking an exam that I wrote. The purpose of this was to see how much time it took just for the physical act of writing (correct) answers, in the ideal case in which the answer is immediately known. The amount of time available for students to take the exam should of course be greater than this, but greater by how much? For certain courses, I developed a simple formula and adjusted the number and type of test questions accordingly.

I haven't done that in a while. Perhaps I am in that dangerous stage of my teaching career when I assume that I 'just know' how to do things like create an exam that is fair in length and level. I like to think there would be some warning signs (in my teaching evaluations? in other student comments?) if I have gone astray (or were to do so in the future).

I have started grading the exams turned in early. Students in previous classes have told me that it stresses them out if I start grading while some of them are still taking the test but it's not as if I am chortling in an evil way as I slash giant red X's through incorrect answers. I am not even sighing. I suppose it could be disconcerting if I also made happy sounds while grading. So I try to be subtle and quiet, serious and respectful as I make my first forays into grading amidst continued test-taking by the remaining students. Do you start grading turned-in exams whilst other students are still taking the test?

.. with a few minutes left to go, ~50% of the class remains. Some are just staring at the exam paper, some are writing rapidly. Some are obsessively clicking their writing utensils.

The deluge of exam-turning-in is about to begin. Some students acknowledge my presence as they turn in the exam and some do not make eye contact. I don't try to read anything into this. Some of the no-eye-contact students may have done quite well and are still feeling the residual effects of exam-stress. My general (delusional?) impression is that the students found the exam to be reasonable. I do not sense major unhappiness or anxiety.

We are in the final minute; 20% of the class remains.

Has anyone ever studied what % of students stay until the very end of an exam time, even if they are finished? Is there a universal value? Or is there a characteristic value for each professor, and/or for particular types of classes, institutions, etc.? The next time you give an exam, please record (and post in a comment) the % of students who stay until the very last possible moment (and was this in a large, medium, or small class)? Do you think the % who cling onto every second avalable is a function of your exam-philosophy or something else?

In this case, 8% of the class needed the exam extracted from them with some effort on my part. I do not like that. What do you do when you have to pry an exam away from a student when time is up? Do you just take it? Do you loom over them? Do you try increasingly forceful statements? Do you beg them to turn it in? Tell them you will not accept the exam if it is not turned in now? Do you walk out of the room?

The exam is done. I was interested in doing some grading while the exam was going on (efficient use of time! good way to see how some students did on the exam!), but now I am not.

Only one student has asked when I would have the exam graded (answer: "in a few days if possible"). I did not mention that cats are an essential part of my grading ritual (TMI).

Dealing with Disrespect

A reader wrote with a quandry about teaching students who are openly disrespectful. I have modified the e-mail message to remove some possibly-identifying details.

Some background: I'm a recent PhD graduate in my late 20s, working as a postdoc in a science field. The postdoc includes teaching one class a year, and I get to initiate and design the class based on my research area, which is really great. 

I've already taught this class once and it went over well. The class at that time didn't count for the major, so it was small and everyone who was taking the class was genuinely interested in the subject. I built a rapport with the students, and ended up with great evaluations that way exceeded my expectations.

This time, the class counts for the major, and my enrollment has increased to over 40. This is good, but now I have a variety of students, at least some of whom are only taking the class to complete their major.

While the students are actually better prepared on average than last year's class, they're extremely demanding -- not just about getting help understanding, which I appreciate and am happy to oblige -- but on details of how I run the class: they want me to make detailed powerpoint slides for every lecture, write practice tests, provide test outputs for their programming homework, and so on. I'm already stretched extremely thin, and besides, I don't want to spoon-feed them! My lectures are actually clearer than when I gave them last year -- and there were no complaints then -- so I don't think my teaching style should be causing problems. They also object to how I demand some degree of class participation and cold-call people for answers. 

The biggest problem I have, though, which is the real reason I'm writing to you, is that I unlike last year, I don't feel I have the respect of the majority of the students -- specifically, many of the male students. I lucked out with a niche set of students last year, but I get the sense now that many of the students have a preconception about my intelligence and ability. For example, today, I made a minor error running through the details of an algorithm. I clarified it quickly and the rest of the class went fine, but several people snickered or rolled their eyes when this happened and continued well after we were back on track. Obviously, errors and confusion in lectures are frustrating for the students, but I'm concerned they're expressing it by snickering so openly. There have been other instances -- for example, students being quick to assume I've made a mistake in writing out a formula, when it's really them misreading it.

Do you have any suggestions on gaining their respect while not being authoritarian (which I anticipate will also cause problems due to my gender)? I am very confident in my knowledge of the material, and fairly confident with leading a classroom, and these reactions are disturbing.

This reminded me of some experiences I had in my early years of teaching. I admit that I never dealt with this sort of situation in any active way. I did try to convey the impression that I was aware of the disrespect (not clueless about it) and was unimpressed by it, and I always hoped that by being calm and teaching the class the best way I knew how, the jerks would get a little bored. Sometimes this mostly worked. 

 

I think being relentlessly calm (without losing your sense of humor or passion for the subject you are teaching) is very different from being authoritarian. You can be authoritative without being authoritarian.

The only thing that really worked for me was to get significantly older than the students. That is not very satisfying as advice though.

Does anyone have any better advice? Have any of you dealt with this type of situation and found a way to silence the snickerers? Does being authoritarian work (assuming you can pull it off effectively)?

 

Room With A Table

To state the obvious: not all classrooms are created equal. Students may have strong feelings about their classrooms (types and arrangement chairs, writing surfaces, boards, screens; sight lines, acoustics, lighting etc.), and professors do as well. And although there are certainly rooms that are better than others, what works well for one class might not work well for another so any one classroom might be good or not good depending on the class/professor.

Several times in my teaching career, I have requested and been initially assigned a "good" classroom (one that works well for the class I am going to teach), only to be reassigned at the last minute to a "not good" classroom. To the casual observer, the differences in these rooms may be quite subtle, so I may seem like an unreasonable complainer when I object, but a room with chairs in rows is very different from a room with chairs around a table. A room that is a 12-second walk from my office is very different from a room that is a 12-minute walk from my office. A room with projection equipment is very different from a room with no projection equipment, and a room with a giant touch-screen TV is very different from a room without.. and so on.

You may have guessed that a classroom reassignment happened to me recently, and your guess would be correct. Another annoying thing about this late reassignment is that I had spent some time over the summer specifically preparing teaching activities for the room to which I had been originally assigned. Much of this time was wasted because my actual classroom does not (and cannot) have the same features as the original room.

A classroom re/assignment is not a neutral thing; just because a certain room will fit the number of students enrolled in the class does not mean that the class will "fit" in that room.

But I don't want to be (too) cranky so early in the new academic year. I am disgruntled about this particular issue but overall quite excited about teaching one of my favorite courses.

Of Course It's True That Professors Grade Easier Than TAs

Last week when I was in a cafe waiting for my mediumskimicedmocha, I overhead one student say to another, "Of course it's true that professors grade easier than TAs", and the other student agreed with that statement. 

Of course! I rather liked this indication that we professors might actually become nicer with time, as opposed to more cranky and mean.

But do you agree with these students? (ignoring the 57 million variables for which we cannot scientifically or otherwise account in discussing this issue now in this blog post and comments).

Some considerations:

- If you used to be a teaching assistant and are now a professor, assuming that you have even a shred of objectivity about this issue, do you think you are an "easier" grader now than when you were a TA?

- If you are a professor now and you teach a class with teaching assistants, do you think you are an easier grader than your TAs? Is this generally true?

- If you are a teaching assistant now, do you have any idea how your grading "hardness" compares with that of the course instructor(s)?

Over the years, in some classes I have been an easier grader than my TAs and in other classes I have not, but if I had to generalize over my career, I would conclude that (1) I am an easier grader now, as a professor, than I was when I was a TA, and (2) I am commonly (but not in every case) an easier grader than most (but certainly not all) of my TAs. I gauge the latter by how many complaints I get about TA grading and, when faced with a grading dispute, whether I think the TA assigned a reasonable grade or was too harsh. [The latter case creates the tricky situation of needing to be fair to the student without undermining the TA, a topic for another day.]

There are likely many explanations for the TAs-are-more-severe-graders phenomenon, but some obvious ones that spring to mind are:

- We are more idealistic when we are just starting out in a career. We have standards, and these are not as flexible as they become later, when we have been teaching for years and might be more willing to reward a glimmer of knowledge as opposed to being severely disappointed that an answer is not as correct or complete as it should be. That does not necessarily mean that we old(er) professors are jaded and have lower standards (though it may).

- At least at the beginning, when we haven't had much experience as a teaching assistant, we don't have much of a basis for comparison and perhaps not much perspective to guide us in the more subjective aspects of grading things involving writing and equations and diagramming. When I was a TA, it was the rare professor who provided much guidance about grading issues such as partial credit, so I mostly made it up as I went along. I figured/hoped that as long as I was consistent, I couldn't go too far wrong.

- A related explanation: Some inexperienced TAs don't have the confidence to give partial credit for partially-correct answers. I recall a time -- many years ago -- when I (the professor) provided a TA with a detailed answer key to an exam. Fortunately I looked over some of the graded exams before handing them back to the students because I ended up having to re-grade several questions entirely because the TA had been inexplicably harsh. For example, in the answer key that I gave to the TA, I had indicated that the correct answer for one question was something like "kitty cat". That was the complete, official name of the thing that was the answer to the exam question, but it did not occur to me that the TA would give students no points if they only wrote "kitty". I should have written on the answer key that "kitty cat" or "kitty" or "cat" were acceptable for full credit, but it didn't occur to me that the student couldn't deal with this level of variability in student answers. Anyone who wrote one of those words clearly knew the answer, so why take off any (or all) of the points? I think the TA just lacked the confidence, and for some reason didn't even want to ask me about it while he was grading.

Now I am wondering: Assuming that I have become easier as a grader with time, have I plateaued or does the grading-easiness trend continue with time (and with what slope on a grading-easiness vs. time plot)?

Annoyance Avoidance

It comes as no surprise that (according to my unscientific poll) the two most-disliked questions that students ask professors are:

Did I miss anything (important)? and Is this going to be on the test? (and variation thereof)

And yet, clearly students want to know the answers to these questions. Is there a way for students to get the desired information and avoid annoying their instructor?

Probably not. At least, not without doing a bit of work first.

As I mentioned in an earlier post, I highly recommend that students take whatever steps they possibly can to answer the first question on their own, and then approach their instructor with specific questions about the material. If a student comes to me and says "I missed class last Tuesday but I have read the relevant chapter in the textbook, looked at the review material you posted online, and read [my classmate's] notes that s/he took in class that day, and I just have a few questions..", I am totally happy to answer those questions.

For question #2 and its ilk ("Do we need to know...?"), a similar approach of asking specific questions about course material may be the way to go; that is, by asking substantive questions that show some thought. I realize that is not the same as asking whether something is on a test, but I think that a thoughtful approach to question-asking might result, in some cases, in the gleaning of information such as "I don't expect you to know that particular topic in that much detail" or "Yes, that's an extremely important topic". But again, some work (by the student) is required to get to that type of conversation.

As I was reading the comments and assembling the polls, it occurred to me that some of the items listed used to bother me more than they do now. Am I mellowing with age? In particular, I don't mind the "Is this going to be on the test?" question as much as I used to. It is a familiar and routine part of the teaching experience, and I am happy to roll with it and give a sincere answer, particularly to intro-level students (less so with majors). However, I have not yet achieved a happy coexistence with the first question, perhaps because that one bruises my delicate professorial ego and the second question does not.

In terms of the other items in the List of Annoyances, it is clear that the issue of greetings in e-mails and in person is a minefield. I think it would be very useful if new-student orientations provided guidance on this, as there is huge variation from institution to institution. There is also variation within institutions and we can't expect our intro-course (non-major) students to know the culture of our department/unit. It is probably a good idea, therefore, if students start with the most formal mode of greeting ("Dear Professor X" in e-mail; "Professor X" in conversation) and see if they can pick up on any clues whether it is OK to be more informal. It is probably always a bad idea to refer to men as "Professor" and women as "Mrs/Ms/Miss/firstname" by default.

Professors can also help with this: In the first day of class in my intro-level courses, I specifically discuss the topic of how I want to be addressed.

But now I would like to explore this topic of mellowing-with-age a bit more, not with a poll but just with a request for comments. If you have been teaching for at least a few years: as you survey the list of Annoying Questions (and maybe others not listed), think about whether your feelings about these questions have changed with time. This question does not apply to anyone who has never been annoyed by any of these questions, ever, but for the rest of us: has your annoyance level (whatever that is) decreased, increased, or stayed the same with time?

What Am I Missing Here?

This has surely been written about 57 million times before, but somehow it would be nice to get the message out to students about the perils of asking a professor the following question about a missed class:

Did I miss anything? or
Did I miss anything important?

How can we broadcast the information far and wide so that this question will never again be asked in this way? Is that asking for too much?

In fact, when my teenaged daughter heard her parents discussing this recently (we had both been asked this very question in this very way), she was a bit stunned, having asked a somewhat similar question of teachers in the past and intending no offense. Despite being the offspring of two parents who have infused her with secret professorial knowledge since the moment of her birth, she somehow escaped the knowledge that this question is considered offensive by sensitive professorial souls.

Her response, which is likely common to many students who ask this question sincerely, was: But sometimes my teachers don't say anything important during a class.

OK, understood. Ouch, but understood. But: if that is true, are you really going to get a useful answer out of aforementioned teacher if you ask them this question? So why not rearrange the words slightly, avoid causing offense, and maybe get some useful information?

So then we had an intense family discussion about how you should and should not ask that question. The difference between what a teacher might consider acceptable vs. not acceptable apparently seems subtle to some (students) even though others (teachers) think the distinction is obvious.

Examples:

Asking: What did I miss? = good*

Asking: Did I miss anything (important)? = bad

* Well, not really. By "good", I just mean 'not as offensive as the other statement'. This question at least assumes that something was missed. And yet, professors may not like this question because it is so open-ended. When asked this question, I tend to reply, "Did you look at the review materials that I posted after class? Did you read the relevant part of the textbook? Did you get notes from a classmate?" The answer is typically no, so then I say "I recommend that you do those things, and then I would be happy to answer any questions you may have about the material." etc. This is just part of the normal day-to-day interaction of professors and students and is OK with me.

But anyway, back to my original question, which I hope you didn't miss: Do any universities/colleges include topics in their 1st-year student orientation sessions along the lines of 'how to communicate with professors' (writing and speaking)? This might be a way to give some helpful hints about such things.

What would these hints be? I am sure most professors could quickly come up with a list of their top 3 or 5 or 10 pet-peeves that are easily avoided. I wonder: can we collectively come up with a short list, or would it actually be a very long list (perhaps with conflicting ideas about dos and don'ts) because we are all such unique individuals with our own special eccentricities and so it is essentially impossible for students to avoid offending us? I don't know, but I think we should find out. So, if you are a teacher of some sort and want to participate in this important effort:

Leave a comment with your top-n list of annoying things that students make when communicating (in speaking or writing) with you.

Then, depending on the results, I will attempt to do a poll, and then we will know something, perhaps.

TQ

Many years ago when I was a non-tenure-track professor, I was taken aback when I learned that the quality of teaching by people in my position -- that is, all non-tenure track instructors -- was considered a priori sub-standard by my institution because we had no long-term investment in the college.

The Dean said this aloud in a meeting and repeated it in a memo. At the meeting, I thought I misunderstood the Dean, but the memo was unambiguous. Tenure-track and tenured faculty had a commitment to the college and the rest of us didn't, so their teaching was, by definition, of higher quality than ours. I doubt if this conclusion was supported by any data; it seemed to be more a belief. I didn't see the point of stating it so explicitly, as a generalization, but of course the focus was to make the the 'real' faculty feel good.

I found that somewhat demoralizing. I cared a lot about my students and my teaching, and I was working extremely hard at my teaching, as I also had in a previous position as a non-tenure-track instructor. Some of my tenured and tenure-track colleagues were doing the same, and some weren't. I channeled my annoyance into targeted loathing of the Dean (but not the college, my colleagues, or my students), and focused on my work.

Later, when I was a tenure-track and then tenured professor at a large university, I learned that the quality of teaching by people in my position was considered by some to be a priori sub-standard because we spent so much time doing research, we couldn't be as good at teaching as those who are entirely dedicated to teaching, including non-tenure-track (adjunct/contingent) faculty. This was the view, not so much of the institution itself, but of a broader community (and in particular those focused on college-level science education).

The reason both statements could be made and thought possibly true by some is that the first one involved a small liberal arts college and the second a large research university.

These generalizations are meaningless. There are good, bad, and mediocre teachers in all the possible types of jobs involving teaching at the different types of colleges and universities in the US and beyond.

Here is my own (possibly meaningless) generalization, or belief, based on no data, just observations in the past 30 years as an undergraduate, graduate student, instructor, and the various stages of tenure-track/tenured professor:

Teaching quality (TQ) for an individual or even a group of individuals in similar jobs does not correlate with type of institution or with job title.

If that is the case, then TQ is more a function of an individual's teaching skills and dedication, both of which are somewhat fluid concepts because they can change with time and circumstance, but are still more important than institution type of job title. There may well be some 'environmental' factors such as teaching load, support from the institution etc., but it is not useful to make assumptions about whether someone will be a good, bad, or indifferent teacher based on whether that person is employed at a small college, a large research university, a private institution, a state institution etc. or whether that person is an adjunct or a senior, tenured professor. I don't think this is a controversial statement, but I am still surprised when -- to this day -- I encounter these stereotypes.

The issue that is relevant to students is whether they are likely to encounter more good teachers in certain circumstances than in others, but I am not going to wade into these larger issues, including debates about how we measure TQ, the value of student evaluations of teaching, whether students are at a disadvantage (or advantage) if taught by large number of adjunct faculty, and so on (I have discussed these in other posts),

My point today is a small one, inspired by a recent experience in which someone from the small-college world expressed surprise that teaching excellence could be found at a large university, bringing back some memories and triggering this post.

Starting From Zero, Again

This comment, from a postdoc, on Monday's post intrigued me:

I think it would be depressing to teach a similar course over and over again. It would feel like every year you go back to point 0 and have to start with another set of ignorant students all over again.

In fact, it's not depressing at all. Sure, certain aspects of teaching the same course over and over can get tedious, and that's one reason why many of us make changes to our course content from time to time, but, at least for me, it is not depressing, and certainly not for the reason the postdoc proposes.

In fact, that's part of the fun of it for me: hitting reset, starting over, beginning with a new group of students who don't yet know the awesome things you are going to teach them (this is not the same as being "ignorant"), and then watching them progress through the term. If all (or most) goes well*, it can be very satisfying. Surely there is an annoying analogy involving gardening/farming and the seasons, or something like that?

[* that is, not too many high-maintenance students, no cheaters etc.]

It's the same with advising grad students, and that's not depressing either.

Does anyone agree with the postdoc about feeling depressed (or other negative feeling) about having to start over at "point 0" with each new group of students? Or perhaps people who feel that way won't pursue a career involving teaching(?). And if they do pursue a career involving teaching, will they always feel that way, or see that, in fact, the rebooting aspect of teaching is not depressing? It would be particularly interesting to hear from someone who thought that it might be depressing to start from the beginning again and again, but found that it wasn't. Or vice versa.

Changing of the Syllabus

Thinking about yesterday's discussion of differences in teaching Science vs. Humanities topics, I wondered how often we professors, over time, change the syllabus of the courses we teach, and, if they do change, why they change. By the phrase "over time", I am trying to say that I mean changes over the years, not within a particular term.

I think it is entirely possible, in any discipline, to teach a "classic" topic that doesn't change dramatically from year to year, or even decade to decade, no matter how much the broader field itself has changed. And then there are courses on "topical" topics, that include or focus on concepts or methods that were unknown in the past, perhaps even in the fairly recent past; this, too, could apply to any discipline, even if not to every course within every discipline.

I was thinking about some of my "classic" science courses in this context, and realized that they have actually changed quite a it over the years, not in terms of the core concepts, some of which have remained unchanged for a long, long time, but in the examples I discuss. There are new applications that did not exist a decade ago (or less), and if I taught the course the same way that I did 20 years ago, students would learn the basics but not be aware of how these can be used to do Modern Science.

I don't think I could specify exactly how much these courses have changed with time, but a very rough guess would be somewhere between 20-40%. In some cases, these changes are reflected in the syllabus (in terms of the list of course topics) and in some cases, not so much. That is, the listed topics are the same, but what I say about them has changed a lot over time. I refer here to changes that are driven by advances in the field, not variations I might introduce to keep from being bored with teaching the exact same thing year after year.

The above discussion refers primarily to undergraduate courses and to some advanced courses, but, for me, graduate seminars are a different beast. The topics of graduate seminars that I teach change significantly over time. It would be impossible (or, at least, very unwise) for me to teach the same graduate seminar over the years; things (including the scientific literature) change too much, too fast for that. I could organize a seminar on the same or similar (very) general topic every 10 years (for example), but the content of the seminar would be very different each time. I would think that this, too, could be said for many other disciplines.

Have you ever taught a course that literally could stay pretty much the same year after year, for a long time, perhaps even decades? Or, (more likely, I think) if you teach a course with content that can, should, and does change with time owing to advances in your field, how much does it change (per whatever unit of time is relevant to your situation)?

No Ability to Grasp

In a recent posting on the New York Review of Books blog, Charles Simic, Pulitzer-prize winning poet and professor emeritus of literature and creative writing, states that "Widespread ignorance bordering on idiocy is our new national goal". His evidence for the spreading of ignorance comes in part from polls showing that large numbers of Americans believe certain things that are known to be false about recent and current events, and also from his experience as a college professor: 

Anyone who has taught college over the last forty years, as I have, can tell you how much less students coming out of high school know every year. At first it was shocking, but it no longer surprises any college instructor that the nice and eager young people enrolled in your classes have no ability to grasp most of the material being taught. Teaching American literature, as I have been doing, has become harder and harder in recent years, since the students read little literature before coming to college and often lack the most basic historical information about the period in which the novel or the poem was written, including what important ideas and issues occupied thinking people at the time.

I will not discuss in detail his ideas about the role of the media in spreading propaganda and disinformation, or why so many Americans believe (or are willing to believe) obvious lies about our own recent history (not to mention the present), except to disagree with this point: "In the past, if someone knew nothing and talked nonsense, no one paid any attention to him. No more." There are many examples throughout history (in the US and beyond) in which much attention has been paid to those speaking nonsense from ignorance. Even so, I agree with Simic that current trends are disturbing.

But Simic also writes: "to have it [disinformation] believed requires a badly educated population unaccustomed to verifying things they are being told", which feeds back into his experience of teaching for 40 years. That's the part that interests me because, in my ~20 years of college teaching, I have not seen a shocking decline in how much students coming out of high school know, as he describes in the excerpt above. [I am also ignoring any discussion of college-educated vs. not college-educated people in the context of the current US election-year explosion of ignorance, and am focusing here only on Simic's personal observations of a decline in college student knowledge and intellectual abilities.]

Why haven't I seen this change in how much students know and can learn? Did the change happen >20 years ago, so the biggest decline precedes my teaching experience? I can't evaluate that from my own experience, but I admit to being skeptical that students in the good old days were more accustomed to "verifying" things that they were told. In fact, today I think that many students are deeply skeptical of what they are told, and we professors have to make a concerted effort to be convincing in our explanations.

Or is the difference not so much my relative youth as a professor but the fact that I teach Science and Simic teaches Literature? He writes about how students today don't have much context to understand literature because they don't read much (relevant) literature before college, and they don't know about history, even the history of places where they grew up (he gives the example of New England mill towns). It is therefore more difficult to teach these students.

In Science, we require less context of that sort (particularly of the historical variety), although we expect our students to know (some of) what they have learned in previous science and math classes. As long as they have learned something in their previous science/math classes, I can take it from there and teach them what I want them to know in my Science classes, whatever the level and format of the course.

But Simic doesn't just say that students haven't accumulated certain facts before arriving in college, but that they also ".. have no ability to grasp most of the material being taught". That's different, and, if true, should also apply as much to Science as to Literature, History etc.

I get as frustrated as any teacher does when some students do indeed fail to grasp a concept (or basic fact), despite being given ample opportunity and assistance. Even so, at every type of institution at which I have taught, I have found that every course contains students with a wide range of abilities. A challenge for me as a professor is to teach them all. This has not changed in my 20+ years of teaching experiences at different colleges and universities.

Another possible explanation for my failure to discern a decline in student abilities over the years is that I am oblivious. I am skeptical about this explanation, but I do not reject that possibility out of hand. I think when we are intensely focused on something for a long time, we can lose perspective. Perhaps I have been "dumbing down" my courses bit by bit over the years, progressively adapting to the declining abilities of my students, so any decline in student abilities was imperceptible to me (?). However, if I had taught the same course in 1992 and then not again until 2012 (assuming that my teaching abilities remained the same and it was a course that could be taught the same despite the passage of decades..), perhaps I would see a difference (?). Somehow, I don't think that is the explanation.

At the moment, I am intrigued by the Science vs. Not Science possibility. I wouldn't want to take this so far as to say that teaching (and learning) Science requires less of an intellectual engagement with, or curiosity about, the rest of the world, as compared to history, literature, and so on. I don't think that is the case. But perhaps some fundamental aspects of (college-level) Science can be "grasped" more easily (that is, with less background information) than, say, a 19th century novel? For example, in this context, is it easier to explain the second law of thermodynamics than the poems of John Greenleaf Whittier?

What do you think? Have any of you long-time teachers seen any change, for better or worse, in overall student ability to understand what you try to teach them? Please give the time-frame of your observations (in years), your academic discipline, and any ideas you have that might explain your observations.

High-Maintenance Equation

A colleague and I were discussing the teaching of large classes (~100 students) and how/whether this is substantially different from teaching a class with 30 students or 300+ students. This is not the first time I have had this conversation in my life, but it is always interesting to hear what others think.

Of course part of the answer to the question of whether there is a difference relates to the structure of the class and whether you have grading support etc., but to the extent that we can generalize, what are the main factors in similarities vs. differences in teaching a lecture-format class of 30 or 100 or 300 students?

Aside from issues such as ability to learn student names, we agreed that a rather major factor affecting the experience of teaching is the number of high-maintenance (HM) students in a class. In a large class, there is a greater chance of encountering HM students, and more of them.

What are examples of HM behavior? It is important that I explain that I do not include in this category students who ask a lot of questions unless the questions are repeatedly and insistently of the 'just tell me the answer so I don't have to think about it or go to class or read the textbook' or similar variety. Just asking a lot of questions does not by definition make a student HM. (See also discussion last year of how HM student behavior is not unique to students; professors do many of the same things when in the role of 'student'.)

Lots of unreasonable requests (for things the student could easily look up on their own), lots of whining and excuses, and/or frequent begging for a better grade are possible components of HM behavior. Students who require a lot of help are perhaps technically HM in some ways, but if they are working hard to understand the course material and are sincere, then I would put them in a different category.

So, just considering the most difficult, soul-destroying type of HM students: Is their effect on your energy level and emotional state diluted by the large number of non-HM students in a class? I would say no (mostly). The type and frequency of HM behavior and the number of HM students are the major factors, not the overall class size. This is a hypothesis that I propose, for discussion.

That is, if you have a class of 100+ students and 1 very HM student, is that somehow less of a drain than if you have a class of 30 students and 1 very HM student? And if you think it is, can you write your personal equation that relates total class size, number of HM students, and their cumulative effect on you?

Don't Try This At Home

For various reasons, the school my daughter attends this year was unable to organize things so that she could be in the math class that followed from the math class she had last year. In fact, the class she was put in this term is working on math that she did 3 years ago.

Fortunately, the teacher of that class does not make her (re)do that "old" math with the rest of the class. During class, my daughter sits by herself and works on her own.

So what does she do in math class? She does math problems that I assign her, based on the math that I teach her in the evenings at home, using an online textbook. That is the temporary solution we worked out with her school: I will teach her math at home.

Why me? Why not her dad?

Because this is what dad-as-evening-math-tutor would be like, we feared:

In contrast, this is what mom-as-evening-math-tutor is like, in theory:

 (though perhaps a bit more alert, most evenings).

So, I am the designated parental math tutor, and here is what I have learned so far:

- The things I hate about grading still apply. Grading doesn't become more fun just because you are teaching your own beloved child. That is, just because I am teaching my daughter, who is the light of my life and a truly wonderful human being, doesn't make it any less annoying when she turns in a messy page of homework covered with incomplete erasures and crossed out things and a mystifying sequence of answers in no particular order (and no helpful labels).

- For me, Science is easier to teach than Math. In Science, I know how to explain things. In math, some things can be explained by examples -- perhaps many examples of different sorts -- but some things just are. That is showing my limitations as a math teacher, something I also encounter when I teach a quantitative Science course: I explain why I am doing the math in terms of the Science, but I don't typically explain the math itself. I just do it.

- There are a lot more (imaginary) people in (this) Math textbook than in (my) Science textbooks and I don't like some of them. Most chapters of the math textbook we are using describe an impressive array of enterprising teenagers figuring things out involving math. That's nice -- I like the textbook quite a lot, actually -- but I wonder how much the involvement of people -- even imaginary ones -- affects math-learning. That is, are we each influenced by whether we relate to the imaginary people and their imaginary problems? For example, I am not so interested in Josh's questions about the operation of his remote-controlled car or Delores' attempt to figure out which phone plan to get, but I am intrigued by some of the scientific and sociological datasets and the various things we can learn by analyzing them. And, although I do appreciate the real-world examples, sometimes I get tired of all these perky teens and just want to play with the equations.

- When you teach math at home, in the evening, to your child, you can have ice cream during class. 

Anyway, despite my shortcomings as a math tutor, we seem to be doing OK with our math-with-mom-at-home arrangement. Even so, once the schedule is fixed so that she can join the right math class at school again, I will happily hand her (and the grading) over to a real math teacher.

How Nice Are You?

In 2008 at this time of year (that is, the Thanksgiving holiday in the US), I had a post about whether or not I give exams just before a major holiday break (I do not). As usual, there were some interesting comments, but I think this question deserves a poll in 2011.

In this poll, a "No" answer implies a deliberate choice to not give an exam or quiz before a major break (you can explain why in the comments).

An answer of "Maybe" implies that you don't really pay attention to holidays/breaks and you just create the schedule however makes the most sense for the class/topic; if a quiz falls just before (or just after) a break, so be it.

A "Yes" answer implies that you deliberately schedule exams just before a break because .. (explain in the comments); e.g., this ensures attendance, you'd rather give an exam just before than just after a break, you are evil etc.

Do you give exams just before a major holiday break?
	No, never
	Maybe, if that's the way the schedule turns out
	Yes, very often or always
pollcode.com free polls

Complaining Early & Often

While serving on a particular review committee at my university, I have seen many examples of negative comments in teaching evaluations for problems that seem like they could easily have been fixed during the course if only the professor knew there was a problem (and was willing to recognize it as such and change something about their teaching style).

Sometimes we professors can sense that something is wrong or that students are unhappy or confused. Some students will tell you directly, but most try to express their unhappiness and/or frustration in unspoken ways. Unless you ask them what the problem is, perhaps even by doing a mid-term evaluation to get anonymous comments, it can be hard to know what the problem is in some cases. Of course, if you just handed back a test and the average score was 17%, you might have some clues as to why students are unhappy.

The situations I am thinking about don't have to do with difficult tests, but more with teaching style. I have talked about some of these issues before, such as pacing vs. being stationary, having an accent and/or speaking too fast, using various formats and devices for writing, projecting etc.

Whenever I see a file with very negative teaching evaluations, I always wonder if any students complained somehow, to someone, during the course. Some of the problems seem entirely fixable during the term, when there is time for the students to benefit.

I tend to assume that no one complained because I seldom see a comment like "Although we told Professor X that we could never see his writing when he used the red marker, he kept using it." Instead, it's more common to see the complaint "I could never see the writing on the board when he used the red pen."

I can certainly understand why some students would not want to complain directly to or about a professor during a class. What if the professor gets angry and punishes them by giving impossible exams and low grades? There is surely some anxiety about the consequences of complaining.

Of course, there is a difference (or can be, anyway) between complaining and making a request. That is, instead of "I hate it when you use the red pen", something along the lines of "It would help us all see your writing on the board better if you only used the black and blue pens."

Other problems, of course, are more serious and more difficult to fix during a course; for example: comments about a professor's disorganization, inconsistency, perceptions of unfair tests or rude comments, refusal or inability to provide clear explanations or answers to questions. In those cases, what can a student do?

Don't wait for the teaching evaluation and don't be satisfied with writing negative comments on some professor-rating website. Get organized: talk to other students, find out what the issues are, get examples, and write it all down. Then ask an undergrad advisor, respected senior professor, or relevant administrator what to do. If the complaints/requests are reasonable, perhaps there are faculty or administrators who can pass along suggestions aimed at fixing problems in time to help the students. In some cases and at some places, these concerns will be dismissed or ignored, but I think it's worth a try.

In some cases (but probably not the extreme ones), there might even be a reasonable explanation for what seems like bad teaching. I one case I can think of, a professor used a lot of text-heavy slides in a class. The students complained about it in the teaching evaluations, but not one of them had mentioned during the course that they hated this. It turns out that there was a hearing-impaired student in the class, and the professor had been asked to put a lot of text on slides, and had spent considerable time doing so, out of concern for the hearing-impaired student.

There are ways that this particular situation could have been dealt with better by the professor and the students. For example, the professor could have explained what was going on, and could have found a way to present the course material without making the students feel bludgeoned by text-laden slides.

Of course I wish the major teaching problems could be fixed, but it is these easily fixable problems that I am focusing on today because they are fixable with a bit of two-way communication between professors and students.

So, student-readers: Are there any possibly-fixable issues in the classes you are taking now that you wish could be fixed during this term? If you give us some examples, the professor-readers can comment and say "Yes, you should definitely tell your professor about that" or "No, don't do it" (but here's a suggestion for dealing with it). More likely, you will get both answers for any particular example, but the results could be interesting anyway.

Everything I Say Might Be Important

The other day, while working intensely on editing a document in a cafe, my attention wandered for just a moment (or two) and I tuned into a nearby conversation between two undergrads. One of them said something like this:

I hate Professor X. He never tells us whether anything he says is important, so we have to assume it is all important and so I need to take notes on everything he says and it is impossible to do that.

There was a bit more to the conversation, and it seems that this student is really struggling to sort out what is important and what is not important in a lecture, and therefore isn't getting much out of the class. If a lot of information is presented in each class, and all of it is theoretically equally "important", she can't possibly take it all in, she spends a lot of time feeling frustrated, and she therefore hates the class (and its professor).

One thing that interested me was the idea that it's better to assume that everything might be important than that none of it is (or might be). A famous gripe of professors is when students who miss a class or arrive late ask the professor if they (the students) "missed anything important". Would you rather have students wonder if you happened to touch on anything important in a particular class, or would you rather have students assume that you are probably saying lots of important things (if only they knew what they were) and hate your class as a result?

Not knowing anything about the professor or class or student in question, I am not going to speculate more about that particular situation, except to say that I hope that the student goes to talk to the professor -- not to complain, of course, but to discuss strategies for understanding the lecture material and thereby to alert the professor that at least one student is confused and struggling.

Overhearing this conversation made me think about how/whether I convey **What Is Important** and What Is Not, while teaching in a classroom.

Most of us are well aware that some students classify information as important (likely to be on the test) vs. not important (safely ignored). And many of us professors think that everything we say is in fact important (and could be on the test), but realistically, some of our pearls of wisdom are more pearly than others. We may emphasize a certain topic (perhaps even specifically described as important, key, critical, significant, fundamental etc.), and then some/most of what immediately follows is elaboration: we give examples, we try to explain and not just state. All of that is important, but there is also sort of a hierarchy.

I am going to try to be more aware of this in my classes, but I am curious as to how often I say "This is a really important point/concept/question" (and then, ideally, explain why it is important and not just assert it). I think I try to be clear about this and not just assume that students will automatically know, but to be sure, I do provide review questions that represent important information from each class. Or, I should say, the most important information because, of course, it is all important.

Loaded

Some colleagues and I were talking about teaching loads over dinner recently. First, let me say that I do not think that the word load implies that we find teaching a burden. It's just a term that means "how much one teaches per unit time" (term or academic year).

I hope that longtime readers have the impression that I like and value teaching (and students), as that is, in fact, how I feel about teaching (and students). That does not, however, mean that I want to teach a lot. By a lot, I mean more than 2.5 classes/year. OK, maybe 3.

If I teach a lot -- and there have been years when I taught > 3 classes -- I like teaching less because I don't have time to focus on doing it well. I don't have time to give my students the attention they need and deserve. Excessive grading erodes my mental and physical health.

If I teach more than 3 classes in an academic year, I don't have as much time for my graduate students and postdocs and undergrad research students, and it's more difficult to find time for research, papers, proposals, conferences, and all those things that are the other major component of my job. I have a very active research program, and it requires a lot of time to keep that going.

Although I am an epic multi-tasker and am happy work late into the night and on weekends, there are limits to what I can do in terms of doing both research and teaching well, and my personal limit -- given the size of my research program and group -- is about 2.5 classes/year.

Fortunately, that happens to be my average teaching load. It doesn't have to be 2.5 classes every year. It can be 3 some years and < 2.5 other years; that's fine. It's good to have flexibility.

Yes, I know that other faculty teach 4 (or more) classes in a year and also manage to get other things done (research, life), but I am not one of those people who can do all of that well.

Are my university and the people it serves getting their money's worth out of me? How would I fare in an evaluation of my usefulness to the university? That depends on the factors in the equation. I have brought in a lot of grant money, advised and graduated a lot of students, published a respectable number of papers, and received high teaching evaluations. I would fare well if those efforts are considered.

I would not fare well if the number of courses taught/year is a major factor (although my teaching load is not out of line with my department or similar departments). The low number of courses (relative to, say, humanities faculty teaching loads) might be somewhat offset by the fact that I teach some large to moderate sized courses, but there's no getting around the fact that many of us science professors teach less than our colleagues in other units of the university.

Should I teach more? Would it be better for the university overall if I taught more and did less research and advised fewer graduate students? I don't know.. that's a loaded question.

Overexposed?

Today in Scientopia, I discuss a reader's question as to whether it is better to take a 1-year, non-tenure track, teaching position that will/might become tenure-track (and thereby possibly getting the 'inside' track on the TT job), or whether it's better to wait and apply for the tenure-track job.

Magical Teaching

As an advisor, I spend a lot of time explaining to students how to do research. I am continually reminded about what is reasonable vs. unreasonable to expect someone to know. Every year, there are new students, so the slate is wiped clean, and certain things need explaining from scratch.

I have shared teaching materials with colleagues before, but I realized recently that I had seldom explained to someone else how I teach, and yet I had to do just that because someone new to teaching will be teaching a course that I have taught for a long time. I gave this person all my teaching materials -- syllabus, image files, review information, quizzes/tests, labs etc. -- but when we looked at some of it together, I realized how lifeless these teaching materials are when they are completely disconnected from the classroom experience.

For example, I can look at a list of topics and remember my explanations and examples and the questions I asked the students and what their responses were and how we discussed these, and what happened next, and so on. That's the invisible, magic part that you can't see just from looking at lists of topics or even some figures.

I don't mean magic as in pulling bunnies out of hats, although perhaps there is a bit of that sometimes (particularly in large intro science classes), but more in the sense of something that isn't very scientific, even if you are teaching Science. It's the feeling you get when you are teaching -- the energy, the interaction, the information you get from how the students respond (mostly non-verbally) to what you are saying.

The way I learned to teach was by going to the class of an extremely effective professor for whom I was a TA. I had had excellent professors as an undergraduate, but I was learning other things from them. In grad school, I was a TA and I needed to learn to teach, so I went to watch this person -- a legendary teacher -- teach a big intro science class. I went to every class, and I even sat in on the same class with the same professor for more than one term.

Some things did not translate from his teaching style to mine. He was (and is) a charismatic man. I am a profoundly uncharismatic woman. Somewhere in the FSP archives is an anecdote about how he successfully used a particular method to quiet the multitudes in the giant lecture hall. Years later, when I was teaching the same class as an instructor (just after getting my PhD) in the same lecture hall, students complained that I was treating them like children when I tried to get the class to quiet down; I reminded them of a kindergarten teacher.

The other professor had never gotten this comment, ever. Same method, different people: effective for him, not effective for me.

But a lot of his methods did translate well, and for many years I emulated his teaching style until I felt more confident and started developing my own style.

More than 20 years later, my teaching mentor is still my teaching mentor, and has been visiting me recently to talk about.. teaching, pedagogy, and sports. Or, at least, two out of those three.

Will There Be Anything Else?

One of my colleagues has a teaching philosophy that contains the following principle:

If you do a lot for your students in terms of providing 'extra' materials (study guides, lecture notes etc.), they will ask for more and more and more and be unhappy. If you don't provide them with much -- maybe just some review materials before an exam -- they will be content with what little you give them.

This is a bit cynical, so I hasten to note that my colleague is a great teacher. When he teaches a large intro science class, students applaud at the end of the last lecture. Sometimes when he is in a near-campus coffee shop, former students have the barista send drinks and treats to his table with their thanks. Students know he cares about the course, even if he pours most of his energy into the in-class part and not so much to other parts.

I have always disagreed with him about the "give them an inch.." hypothesis, but I must admit that every once in a while there are some data (or a datum) to support it.

At the moment, I am thinking about a comment on my recent teaching evaluations.

First consider this: After each class, I post questions that cover all the main topics of that day's lecture and I provide a pdf of the presentation file if I showed images during class. During the term, I give the students all the exams from the previous year's class. And, in a new development for me this past term, just before class time I post a file with the images I will show in class that day. I annotate the images in class, and the students who bring electronic devices to class can do so as well if they want. For those who don't access the image file in class, I suggest that they note down the slide numbers in their notes so that they can later match images to notes.

What I don't do is provide the image file well in advance. I am typically tinkering with the images until the last minute, and I explain this to the students when I describe what course materials I provide/don't provide and why.

And yet, despite providing quite a lot, one student wanted more. Not only did this student want me to provide the presentation file (with images) well in advance, but s/he also wanted me to print out the file each day and distribute a copy in class so that students could take notes directly on the images on paper.

OK, no problem. Just let me know how many images per page you would like. Should I print the pages in color? Don't worry about the cost. One-sided or double-sided? Of course I will collate the pages, but would you like them stapled or unstapled? If stapled, do you prefer the upper left corner or the upper right? Just let me know if you want me to punch holes in the margin so it will fit in a 3-ring binder. And just so no one feels bad about all the paper this will require, please rest assured that I will try to find paper made with a significant component of post-consumer waste.

And will there be anything else? Oh yes, OK, sure: I will highlight the points that will definitely be on the test. No problem.

This post is an example of how we focus on absurd little comments that are kind of (or very) negative in our teaching evaluations, even if the overall evaluations are positive. In the class I taught last term, 100%* would recommend me as an instructor! 100%* find me approachable and respectful! 100%* think I know my subject well and present it clearly! All of this makes me happy and I am grateful to my nice students who took the time to do the evaluations and express their satisfaction with the course.

Clearly, even the student who wished that I provided handouts in every class was overall happy with the class, so in this sense my colleague is wrong: students may ask for more, but it doesn't mean they are dissatisfied.

So, my typical approach to these situations is to consider all suggestions as constructive criticism, seriously ponder whether the suggestions are feasible, and use those that are and ignore those that are not. I may get ideas for how to explain my teaching philosophy in a different or more complete way (in this respect), even if I don't change what I do. To the extent that these evaluations by students are any use, for me, this is a good use for them.

* of the 89% of students in my class who did the evaluations