Thursday, July 10, 2014

Room for Improvement

Student comments on my teaching of a particular course:

Great professor!
I have enjoyed this class!
I liked the readings.
This course required too much previous knowledge.
Professor very helpful with homework.
Homework very useful for class.
Well-constructed lectures.
Very organized lectures.
She speaks very clearly.
She answered my homework questions.
She provided images and charts to supplement the subject matter.
The in-class exercises were helpful.
I liked the practice exercises we did in groups during lecture.
I liked that she asked questions during class and this helped deepen my understanding of concepts.
Useful supplementary material to help us understand lecture material.
She explained the topics completely in class. Didn't use a textbook as a crutch.
It was great that lecture and lab material were well coordinated.
She was always ready to answer questions.
She was always willing to help with any questions.
She provided the subject matter very clearly.
The last project was too much work for this level of class.
Lecture presentations very clear.
I liked the in-class exercises.
You should improve your teaching methods.

Note that almost all of the comments are in the 3rd person (except for the last one), as if the students were writing to someone else about me, rather than writing to me with feedback. I don't know if it matters in terms of type and level of feedback whether the student is imaging an unknown audience or speaking directly to me (?). At evaluation time, I give a little talk to the class about the importance of this feedback and how it is used by instructors and the department/college/university, but I think there is still general confusion among students about what exactly the purpose of these evaluations is and who reads them and whether anyone cares what they think.

These are overall nice comments, and unfortunately also rather classic in that the criticisms are too vague to help me understand what the specific complaints are.

The last comment, despite being too vague to be useful in any specific way, is absolutely right. Despite being deep into my mid-career years, I don't want my teaching to fossilize. I want to improve. In recent years I have attended teaching workshops and gotten some ideas from those. When I team-teach, a faculty colleague is in the classroom with me, so I get some peer feedback. And last term, I jettisoned the too-long and too-detailed textbook and provided focused readings, including some that I wrote myself. That seems to have worked quite well (or at least no one said they missed having a textbook), so perhaps that counts as an improvement. I would also like to do some new things involving e-learning and have been to some workshops and meetings about that.

I am thinking about teaching because I was just looking at my evaluations, though mostly I am enjoying having lots of uninterrupted time for research. This week I even managed to submit a manuscript on which I am primary author. It's been about two years since I've been able to do that (and I don't mean to imply that I did it alone -- an excellent colleague was essential to the completion of this paper).

As I was finishing the paper (and a related grant proposal) recently, it occurred to me that I could create a new teaching module based on this work and incorporate it into the class for which I just received teaching evaluations (not, of course, as extra work but replacing some older material). Probably more than any major change in teaching style, a realistic way that I can improve my teaching is to find good ways to incorporate new material -- specifically, integrating New Science with Classic Science, so that students learn the fundamental stuff without which they are incomplete as scientists and people and yet are also exposed to new things that help them see where the field is at (including being exposed to unresolved questions that might inspire them).

Anyway, it's been a busy summer so far. My father recently asked me if my husband "also has the summer off" and I was actually quite calm about it this time. Have you had a similar conversation with anyone yet this summer? Parents? Neighbors? Friends? Students? Assuming that you do in fact work in the summer even if you are not teaching, did you (1) smile serenely and let them continue to exist in ignorance; (2) correct them (a) calmly, (b) not calmly; or (3) lapse into stony silence (if having a conversation) or send a glaring emoticon (if in e-contact)? (or other..).






Monday, June 23, 2014

Men are from Pluto

A colleague and I were talking about this and that recently and he said that at some point he needs to find a new research topic, as the one that he has been working on (very successfully, and in fact sort-of pioneered) is getting very crowded. It's not as much fun (says him) to be in a crowd instead of way out ahead.

So then he said that it was difficult to start working on a very-different topic because it can be difficult to get funding if you lack a track-record and expertise in that new thing. True enough. So I said, "Collaborate" (unsaid but well known: That's what I do).

He said, "No, you can't project authority if you collaborate."

Discuss.

Context: We are both full professors and therefore getting adequate credit for our work is not a career life-or-death issue as it is for early-career scientists. For the early-careerers, this can be important (depending on your particular context). Collaboration can still be a significant research component -- enjoyable and rewarding in many cases* -- as long as you also stand out from the crowd in some way for your ideas and expertise.

But other than that, who cares about projecting authority? OK, some people do. My colleague clearly does, and he is very good at it (projecting authority). I don't really care. Well, I do a bit (I don't like being overlooked), but I don't think collaborating has lessened my "authority". If anything, it has increased it.

I reject as a general philosophy the idea that collaborating de-authoritizes you (I just made that word up), although if that's what floats your boat, go ahead and enjoy your authority (alone).



* if your colleagues are not jerks, and if they don't hold up manuscripts and proposals.

Monday, June 09, 2014

Measure for Measure of Success

Something that I have been seeing more and more in grant proposal reviews (my own and those of colleagues who have shared theirs with me) is the idea that it's not enough to have a record of success advising grad students, undergrads, and postdocs in research -- you have to understand and explain your advising techniques and you have to have a plan for assessing and improving.

OK, I get that, but even when I attempt to do those things, it isn't good enough for some reviewers. They think that I (and my colleagues) are relying too much on past success and traditional measures of success (degrees, publications, conference presentations, post-graduation employment). They are not convinced that that is sufficient. They want something different. Apparently, unless you change something, you are not improving and therefore are not being transformative, or something.

Example reviews (comments condensed/reworded to remove any identifying vocabulary):

A highly qualified PhD student has already been identified for this research but the mentoring of this student and an undergraduate is largely assumed based on prior experience of the PIs. The PIs have records of successful advising but should include in the proposal a more intentional discussion of how they plan to train the next generation of scientists. The mechanism for success is not explained and there is no plan for assessing success of their mentoring. How will successful training of the graduate student be determined other than by the record of publications, presentations, and completion of the thesis? Although the research is potentially transformative and this is an excellent team of researchers, because of these shortcomings in the broader impacts I have given the proposal a lower rating.

That makes no sense to me. I am definitely not saying that we all deserve to have all of our grants awarded just because we have had past success. However, I think that if the proposed research is deemed excellent by a reviewer and the PI has a demonstrated record of success with advising, it does not make sense to downgrade a proposal rating for the reasons given in the example review above, contributing to the rejection of the proposal and therefore a lack of funding for the graduate student.

Here's another:

[From a review of a proposal that included one week of salary for a soft-money research scientist who runs a lab in which students would do some analyses for a proposed project]: Description of the mentoring of the postdoc is not well developed. There is no mention of career counseling. Mentoring in professional activities such as writing proposals and papers is confined to discussions and support for participation in conferences and workshops. There is no mention of how the postdoc will be mentored to collaborate with diverse groups of researchers and students. There is no description of the postdoc's career path in the context of developing an effective mentoring plan for him.

And this:

[From a review of a proposal that included a substantial component of support for undergraduate research]: These PIs have a long record of success in advising undergraduate students in research but no evidence is presented for how the field of research on undergraduate research will be advanced. 

These are just anecdotes, of course, plucked from reviews of different proposals by different PIs. At least one of the proposals even involved a colleague who does research on teaching and learning. It wasn't enough. Some of us PIs have attended national and local workshops on teaching and learning, read some of the relevant literature, even co-authored papers (some with education specialists) in science ed journals. It's not enough.

I think that giving attention to effective advising is an important component of research (and therefore grant proposals), but I also think these and similar reviews show that certain reviewers have run amok and are harming the very people (students, postdocs) they think they are helping.





Friday, May 16, 2014

Ab,surd

A colleague recently shared a review of his (rejected) proposal. The most negative review contained this statement:
There were a number of editorial errors found throughout the proposal. Some were missing commas and the like,
The mind, like, boggles.

Now I need to know: have you ever
  • commented on punctuation in a review? (minor errors, egregious errors, pet peeves?) -- did it affect your overall rating of the proposal, as far as you know?
  • received a comment on punctuation in a review? (minor errors, egregious errors, pet peeves?)
And if there were punctuation, grammar, and/or spelling errors in a review that criticized you for real or perceived punctuation errors, how did you feel about that?

Did you think, "Oh well, the reviewer is just bashing out a review on a webform and of course shouldn't be held to their own high standards for punctuation in proposals. Perhaps that missing comma on page 10 of the proposal indicates that I am a sloppy and untransformative scientist and therefore didn't deserve a grant that would have supported student research."

Or did you think something a bit more negative about the reviewer's punctuated hypocrisy?




Thursday, May 08, 2014

Liveblogging the Exam

Although taking an exam is most certainly more stressful than giving an exam, giving an exam can be quite stressful. I am not asking for sympathy, I am just stating a fact.

The most stressful exams to administer are those in large classes in which students are packed into every available seat and there may be (alas, too often) issues with cheating. You can devote considerable time and effort to anti-cheating activities such as giving multiple versions of exams, you can have students sign an honor statement, you can patrol classroom non-stop during the exam, or you can just hope for the best.

Giving an exam to a small or medium-sized class is less stressful because the logistics are easier, but that is not to say that giving an exam even in these circumstances is lacking in stress. Or, at least, that is my opinion. Is there anyone who would rather give an exam than have a regular class? I would much rather have non-exam class time. [I am deliberately not addressing the possibility of not having exams at all. In some courses I do not give exams, in others I do, depending on the course.]

OK, so I am about to give an exam in a medium-sized class. I am not dexterous enough to blog while handing out the exam (and the TA does not seem to be in evidence), but I can combine semi-live and liveblogging to try to capture the essence of the experience from the front of the room.

I enter the room. They are all here, in their seats, staring at their notes. This has been a very punctual class so I am not surprised they are all here on time. In a more typical class, students would appear throughout the first 10+ minutes of the class, even on an exam day, making a lot of noise as they rush in, grab an exam, find a seat, deal with the logistics of finding a writing utensil and putting their water bottle in a suitable place

Put your notes away! This takes a moment. I try not to let it eat into exam time (there is another class in this room right after ours), so I start handing out the exam to the nearest row of students who are note-free.

Can we start now? I should remember to say that they can start as soon as they get the exam but sometimes I forget and then someone always asks. It is not a large class, so the time difference between those who get the exam paper first and those who get it last is about a minute. If anyone in the back needs that extra minute, they can have it at the end. In a large class, I need a fleet of assistants to help hand out exam forms so that no student has to wait too long to get the exam or to get their question answered if they have one during the exam.

The room is never totally quiet. Someone is always turning a page, even if the exam has.. one page. And certainly if the exam paper has 2 pages: rustle rustle rustle. The second-most common noise is erasing. There is a lot of erasing going on at this very moment. Some students have very impressive erasers.

Two students just had questions about different exam questions. Both were answered easily by my pointing to a key word or words in the exam question. In both cases the student immediately saw that the answer to their question was right in front of them and thanked me.

The first student is done with the exam, halfway through the allotted time.

The second student just finished, with 20 minutes left to go. Make that three students.

Are the ones who finish very early the ones who are doing well in the course so far? In fact, there does not seem to be a strong correlation between those who are doing very well or very not-well in the course (to date) and those who finish the exam early, just as I predict that there will be a random collection of students (with respect to course performance) at the very last second of the exam.

At various times in my teaching past, I have timed myself taking an exam that I wrote. The purpose of this was to see how much time it took just for the physical act of writing (correct) answers, in the ideal case in which the answer is immediately known. The amount of time available for students to take the exam should of course be greater than this, but greater by how much? For certain courses, I developed a simple formula and adjusted the number and type of test questions accordingly.

I haven't done that in a while. Perhaps I am in that dangerous stage of my teaching career when I assume that I 'just know' how to do things like create an exam that is fair in length and level. I like to think there would be some warning signs (in my teaching evaluations? in other student comments?) if I have gone astray (or were to do so in the future).

I have started grading the exams turned in early. Students in previous classes have told me that it stresses them out if I start grading while some of them are still taking the test but it's not as if I am chortling in an evil way as I slash giant red X's through incorrect answers. I am not even sighing. I suppose it could be disconcerting if I also made happy sounds while grading. So I try to be subtle and quiet, serious and respectful as I make my first forays into grading amidst continued test-taking by the remaining students. Do you start grading turned-in exams whilst other students are still taking the test?

.. with a few minutes left to go, ~50% of the class remains. Some are just staring at the exam paper, some are writing rapidly. Some are obsessively clicking their writing utensils.

The deluge of exam-turning-in is about to begin. Some students acknowledge my presence as they turn in the exam and some do not make eye contact. I don't try to read anything into this. Some of the no-eye-contact students may have done quite well and are still feeling the residual effects of exam-stress. My general (delusional?) impression is that the students found the exam to be reasonable. I do not sense major unhappiness or anxiety.

We are in the final minute; 20% of the class remains.

Has anyone ever studied what % of students stay until the very end of an exam time, even if they are finished? Is there a universal value? Or is there a characteristic value for each professor, and/or for particular types of classes, institutions, etc.? The next time you give an exam, please record (and post in a comment) the % of students who stay until the very last possible moment (and was this in a large, medium, or small class)? Do you think the % who cling onto every second avalable is a function of your exam-philosophy or something else?

In this case, 8% of the class needed the exam extracted from them with some effort on my part. I do not like that. What do you do when you have to pry an exam away from a student when time is up? Do you just take it? Do you loom over them? Do you try increasingly forceful statements? Do you beg them to turn it in? Tell them you will not accept the exam if it is not turned in now? Do you walk out of the room?

The exam is done. I was interested in doing some grading while the exam was going on (efficient use of time! good way to see how some students did on the exam!), but now I am not.

Only one student has asked when I would have the exam graded (answer: "in a few days if possible"). I did not mention that cats are an essential part of my grading ritual (TMI).











Thursday, April 24, 2014

Self-Check

This is a poll I have done before (4 years ago), and I am interested to see if the results will be different now. I am curious about how often you check your citation statistics -- number of citations, h-index, who is citing you etc. You can sign up to get alerts about this, or you can go and check yourself at one of the citation-counting sites. How often do you get information about your citations?

Whether or not you answered the poll 4 years ago (in April! is there something about April that makes me think about citations?), do you think your citation-checking habits have changed in the last few years? 

So:

How often do you check your citation statistics?
  
pollcode.com free polls 

Tuesday, April 08, 2014

Post-Postdoctoral-Post

More than 3 years ago, I wrote a post about hiring postdocs. This post seems to have been linked to a site that diverts some traffic to my ancient postdoc-post and keeps the comments coming (intermittently). The most recent comments are all negative: I am an insensitive jerk etc.

OK, maybe so. The point of the original post was to give a PI's-eye-view of the topic of hiring postdocs, and any topic related to the selection of a few and the rejection of most has a high probability of making some readers angry. Nevertheless, given limited funds for hiring postdocs, PIs must make hiring decisions. How do we do that in a situation that is not as structured as the admission of graduate students or the hiring of faculty?

Different fields have different cultures regarding postdocs: for example, regarding bread-and-butter issues such as salary and fringe benefits, the rate of success of postdocs seeking post-postdoc employment relevant to their training, and more cosmic issues related to respect and independence in their work. And even in the best of circumstances, the postdoctoral years can still be ones of anxiety and uncertainty about the future.

Anyway, here are some recent comments on that semi-ancient post, with my further thoughts on the topic:
I think you are a cheap employer that is complaining about the workforce, when the workforce is getting minimum wages to do the most likely irrelevant work that you do. You should consider yourself lucky for having a position, so don't whine about postdocs, you would be nothing without them.
Fact: A typical starting (just-out-of-PhD) postdoc funded by an NSF grant in my field and at my institution makes anywhere from about $42k to $62k per year (salary). At my institution, postdocs paid from a grant also get fringe benefits, calculated as a % of salary and paid by the PI (with indirect costs to the institution). I am not making any judgement about whether that salary is decent or abysmal; that opinion will no doubt vary considerably. I will say that this all adds up to a lot in the context of a typical (for me) grant budget.

Opinion: It is quite true that my work is irrelevant. Conveniently, the work of my postdocs is therefore also irrelevant. Nevertheless, we manage to do some interesting basic science.

Fact: I consider myself very lucky to have a position.

Opinion: I still get to whine.

Fact: Over the years, I have wasted a lot of money on unproductive postdocs. I would like to avoid that experience if at all possible (hence the original post).

Opinion: I have worked with (and am working with) some great postdocs. I consider our working relationships to be mutually beneficial.

(Excerpt) Yet many PI's .. think they're "sorting through the good and hard working postdocs". This is of course false. They are sorting through the leftovers after everyone else jumped to industry. Thus finding bad apples is more common...and really the problem with "lazy" postdocs is that you don't have enough funding to support them...not necessarily that they're crap scientists.
This opinion about "leftovers" does not apply to my field. I don't understand the point about not having enough funding to support postdocs and so that explains why some postdocs are unproductive(?). If someone accept a postdoc and the salary is not as high as the postdoc thinks it should be, does this give them license to be unproductive? Isn't that self-destructive, among other things?

You come across as a completely obnoxious, self-righteous professor. Just remember that hiring a post-doc is a two-way street. You may be judging the post-doc, but the post-doc is also judging you based on the quality of your research and your scientific acumen. Unless you are perfect in those areas, how can your expect your post-doc candidate to be perfect as well?
Of course the applicants are also judging whether they want to work with me; this is as it should be. Nevertheless, with a few exceptions (such as NSF Postdoctoral Fellows), I hire postdocs using funds that I obtained, so I do get to choose which postdoctoral aspirants I think will do well with the project, do well working in my research group, and do well developing their career.

In the time since I wrote my original post about trying to avoid unproductive postdocs, I have had similar experiences to the ones I described. For example, there was the time when an aspiring postdoc who wanted to work on an ongoing project for which I am PI assumed that one of my colleagues was the PI and explained the project to me (in person, at a conference) as if I knew little about it. Did he not even look at the project webpage? Perhaps I would have been more sympathetic if he had at least explained the research well and described an interesting way that he would contribute expertise or ideas. Alas, he did none of those things. This is not about expecting perfection; this is about expecting a certain level of knowledge, initiative, and drive.

I may be an obnoxious jerk but I do want to state here that I think it is important to be respectful, supportive (financially and otherwise), and appreciative of excellent, hard-working postdocs. I hope to be so lucky to be able to continue to do so.