Thursday, October 09, 2008

Partial Credit Fraud 2

Much to my amazement, especially after writing the post about partial credit a couple of days ago, my daughter informed me that she gets points from her teacher if she doesn't know the answer to a quiz question and only just rewrites the 'question stem'.

Let's say the question on a quiz is: What are the 3 main ingredients of a peanut butter and jelly sandwich?

If you write nothing, zero points.

If you write: Peanut butter and jelly sandwiches have 3 main ingredients.. you get a surprising number of points.

If you write: The capital of the Slovak Republic is Bratislava.. you get some points, but not as many.

I think I now understand why my university students are so enraged when I give them no points for making some non-answer markings with their writing implements in the space intended for an actual answer to a question.

26 comments:

  1. Holy crap!

    This explains my students too!

    I swear to the Blessed Flying Spaghetti Monster, these K-12 Educidiots are ruining our society with all these self-esteem-stroking anti-pedagogical strategies.

    No wonder so many of my students always toss in a re-wording of the question with their word-salad answers!

    ReplyDelete
  2. The more I find out about how our students have been treated before they get here, the more pissed off I get. We've trained an entire generation of students to need someone else to wipe their butts for them.

    Oh, and it's not just the schools...I've had *two* students tell me this semester that they're struggling in my class because they were home schooled and "have never really had to study before."

    ReplyDelete
  3. Exactly. Many of the bad habits they have are due to years of training.

    Partial credit whining -- because they used to get partial credit for non-answers.

    Arguing with the marks given -- it's benefited them in the past.

    Begging for higher grades -- it got results in high school.

    The two things I hate most (crying in my office, and flirting/deliberately trying to show cleavage) I can only justify according to the same theory. It probably worked in the past. In the example of flirting, the guess that it's worked well at school in the past -- well, it gives me the creeps, so I try not to think about it.

    ReplyDelete
  4. Well, keep up the high standards. Don't let the status quo encourage you to prop up the mediocre or reward ineptitude. There is still room for excellence in education, and on Partial Credit, I think you're right where you need to be.

    ReplyDelete
  5. Yikes! Especially for the second example. I can see some benefit in being able to rephrase a question into a statement - that at least calls for some use of logic. Giving information totally unrelated seems particularly worth zero (or minus?) points.

    ReplyDelete
  6. I should also have mentioned that if, in answer to the PB&J question, you answered "The capital of the Slovak Republic is Winnipeg", you can also get a few points "just for trying".

    ReplyDelete
  7. This is not new: In the 70s when my daughter, now 40+, was in the 5th grade she was bring home work that was given top marks, but was full of spelling errors which were not corrected. I went to her teacher to ask that while I did not want her to be penalised more than her fellows, could she at least point out the correct spellings. The answer was 'Yes, I will have to stricter as she will be going to Junior high, and they are stricter", not you will note so that the pupils could learn how to spell correctly. My response was fairly unprintable.

    ReplyDelete
  8. You know we have very objective way of doing gradings in India. During school year, every 3 year or so, there is a common exam state wise which is called board exam, where students gets a roll number and teacher don't know the students for whom they are grading and students don't know who grade their papers. This keeps things in perspectives, if students have complain about their grades, they can approach for re-evaluation and then their paper is reexamined by some other teacher again unknown to students. This objectivity is needed in all levels in American education, not on daily basis, but every few years at least.

    ReplyDelete
  9. Not surprising though sadly - it was starting to get that way when I was in school...

    Though I have to say to landsnark - becareful about lumping all homeschoolers together - there are probably many examples of kids like you've had experience with - but theres also an aweful lot of them who blow my mind with how advanced they are simply because homeschooling allows you to keep moving at your pace (Read the book Susan Wise Bauer wrote with her mother on homeschooling - she's a Harvard grad who is now a professor and is homeschooling her own kids using a modified form of what her mother used with herself)

    Because when I hear these kinds of stories - I refuse to let my future children stagnate in what is our educational system today - I see where I would have loved to have gotten to study much more academic things as a child and could have if I'd been homeschooled - and I want better than even my own education for my children - but our school systems are getting worse not better.

    ReplyDelete
  10. I tend to make my problems multi-part, so that there's one part that even the most brain-dead should be able to get. That way everybody gets some points....not enough points to squeak by, just enough points so that they can't say they didn't get partial credit.

    A person who fails with partial credit still fails, but I face slightly fewer headaches.

    ReplyDelete
  11. It depends a lot on how much partial credit we are talking about. When I was grading as a TA, (so I already felt bad for poorly worded or ridiculous questions and inadequate time constraints), I would give 1-2 points out of 10 if there was anything correct in the partial answer. So if you did this on the whole test, you are looking at a 10-20% grade--this is not at all close to passing so you must also have plenty of real answers.

    I didn't get any restate the question answers, but I think restating the question should be worth nothing unless it took some thought or understanding of the problem--maybe a bit of credit for restating a word problem as a mathematical equation, for example. Writing something true that is related to the problem might deserve credit in some instances--such as if knowing that fact could have helped you at some point as you were working out the solution.

    A better system (than having your TA try to give partial credit because everyone misunderstood the question )is to write clear, relevant questions based on stated course objectives and give students a very straightforward explanation of how the tests will be graded, perhaps with tips like whether students running out of time should bother to write an outline of how they intend to solve the problem. In undergrad, our professors used to break problems up into parts, and if you got stuck on a, you could make up a (reasonable) number/equation to carry through to part b (or sometimes a usable value was given) and get potentially full credit on part b. The point values for all parts were stated on the test so you wouldn't get stuck and waste all your time on a 2 point problem, even though I would do that anyway because I just need to get the answers every time . . .

    ReplyDelete
  12. I guess that is where some people who are answering questions from journalists went to school.

    "Do I know the answer to the question? No. Well, I can make something else up and look like I know what I am talking about"

    I can see the idea in some answers but most of them, not so much.

    ReplyDelete
  13. This is one of the reasons why I like multiple choice tests. And it is the single biggest reasons why students hate them so much, I am convinced.

    ReplyDelete
  14. I'm reminded of early Algebra classes (8th grade, maybe?) where we were given a word problem. There were point built in to the answer for writing "Let x = the number of sandwich ingredients" or whatever. So you could write just that and get something. You could also write a correct answer without that and lose points. It drove me a little crazy as someoen who wanted to do as much of teh math in my head as possible, but at least it makes a little more sense than the above.

    ReplyDelete
  15. I'm reminded a little of the Irish state Maths exam (part of the leaving-secondary-education exams - I'm not sure if there's a direct American analogue). The questions were divided into parts a), b) and c). a) was always a basic problem which should have been easy, b) was more difficult but still took the form of something you would have done in class every day, while c) required some lateral thinking or knowledge of other topics not directly to do with the question. a) was worth 5 marks, b) 10 and c) 20. So to be getting an A, you needed to be at least able to show a reasonable attempt at c) parts.

    The marking system was transparent compared to other subjects - provided you showed your work, you could still get almost of all the marks for a wrong answer that was worked out in the right way: you were docked -1 for a slip (a minor arithmetic error) and -3 for a 'blunder'. My teachers were always very hung up on 'showing your work' for this very reason.

    With regard to okham's comment on multiple choice tests, ordinarily I love multiple choice tests - but the reason I hated them as an undergrad was because they were negatively marked i.e. 1 for a right answer, -1 for a wrong answer. This was not common practice in other courses or other Irish universities and I'm curious to know if it's done in other countries? I have an upcoming licence exam (a test of medico-legal knowledge to finally qualify as a pharmacist) which is multiple choice but not negatively marked. It is, I think, the least worried I have been about an exam in... (quick calculation)...at least 8 years :)

    ReplyDelete
  16. When I took AP exams, we were taught how to get partial credit on our essays. For example, if you had to draw a graph as part of a 10-point question, you could get up to 3 points for labeling axes and giving the graph a title. High school students are taught to love and strive for the partial credit.

    ReplyDelete
  17. ordinarily I love multiple choice tests - but the reason I hated them as an undergrad was because they were negatively marked i.e. 1 for a right answer, -1 for a wrong answer.

    Well, that seems kind of harsh if grading is on an absolute scale, i.e., if a minimum score is required to pass... and pretty stupid if grading is done on a curve... Maybe I am missing something here...
    Anyway, I have never seen anything like that, and sure as hell it would cause an uproar anywhere I have taught in the US or Canada.

    ReplyDelete
  18. That method sounds like excellent training for politics: the ability to say quite a lot, without ever answering the question seems to be a valued skill in that field.

    Personally, I'd rather see students who were taught to either answer the question, or admit that they would need to do further research to be able to answer it.

    ReplyDelete
  19. I've been grading AP exams for years, and at least in chemistry, the problems are broken into *labeled* parts. So yes, there might be a 10-point set of questions on one reaction or experiment, but the 3-point question about setting up a graph is graded as a stand-alone question, independent of a student's answers (or lack of answers) on the other parts. So that's not exactly partial credit.

    If the questions are written well (which, from the point of view of the graders, means that they are easy to grade, while still assessing the skill or knowledge in question), the 3-6 parts of a 10- or 12-point question are, in fact, independent of each other--no one question requires that another question be answered correctly first.

    Re: homeschoolers, I know some are great and really well-prepared, just as some kids come out of some high schools really well-prepared.

    ReplyDelete
  20. Medical students in the UK take lots of MCQs where -1 is given for a wrong answer. It removes the element of guessing. Say you *guess* a patient has a particular condition, and you give them an inappropriate treatment that might kill them, then you must agree it is better to say you don't know and take further advice before treating them.

    With MCQs this is like leaving the answer blank rather than guessing. Someone could get as much as 25% by random guessing (assuming 4 choices)!

    ReplyDelete
  21. Related to this, I recommend Charles Henderson's paper on grading physics exams: http://homepages.wmich.edu/~chenders/Publications/AJPFeb2004.pdf

    He makes the point that some professors assume students know the answer, and thus take points off only when mistakes are evident. Two students who are equally wrong might be graded differently: the one who shows more work gets more points taken off, because they provide more visible mistakes. If you're not careful how you grade, you encourage students not to show their work. (In Henderson's paper they show two examples of students who get the correct answer through incorrect reasoning. Different professors are asked to grade these students, and in general the one who shows more work gets fewer points, even though there's no evidence that either student has more or less understanding, they're both equally wrong.)

    I know this is the opposite of the original post (which I completely agree with) but it's interesting to think this cuts both ways. We have to be careful not to reward irrelevant information with partial credit, but we also have to be careful not to reward students who show too little work.

    ReplyDelete
  22. I guess I am in the minority here, but for elementary school this is entirely appropriate. One of the most important steps in successfully taking tests, or responding to questions in general is to correctly understand the question itself. Teaching kids to take time with this first step should be encouraged.

    Tests also serve to provide feedback to teachers on what the students know. This practice helps teachers distinguish whether students don't understand the question or don't know the answer. It also can help teachers flag confusing questions, (which I personally find very hard sometimes because I have a different perspective than the students).

    But by the time they are in college, they should be beyond that. I always start my undergraduate classes with a description of all the skills they should have, and probably do have since, after all, they were admitted to this fine institution. They no longer get credit for those skills, it is time to move on.

    ReplyDelete
  23. Well, that seems kind of harsh if grading is on an absolute scale, i.e., if a minimum score is required to pass...

    Yes, that was the idea, the pass mark would have been typically 40 or 45%, we were never marked on a curve.

    ReplyDelete
  24. A pharmacy student who works with me described his recent midterm exam instructions; "choose the answer that is your first preference and also mark the answer that is your second preference." Partial credit is issued for a correct #2 answer when answer #1 choice is incorrect.

    It is frightening to see academia sinking to this level. Apparently, the "we are all winners" stratagem has reached the post-graduate level.

    ReplyDelete
  25. Holy crap is right. I wonder if I would have had straight A's if I had employed the "rewrite the question stem" strategy when I was in school?

    Am I missing something fundamental about the philosophy of grading? Is this an official policy requirement of No Child Left Behind?

    I don't think I ever argued with a grade until I got to college, where it was encouraged.

    I was astonished when our professor told us it was our job to point out any mistakes the TAs might have made in marking answers or totaling up the points, and we were explicitly told to re-total our points on every exam just in case. I did catch a few mistakes that way.

    Before that, arguing over a grade would have been like questioning the Pope. The teachers, where I grew up, reigned supreme and unquestioned. For better or worse.

    ReplyDelete