In the course of some recent discussions about Journal Editing, the question arose as to whether someone's reviewing skills should be a factor in tenure and promotion decisions. In particular, should a tenure-track professor's failure to be a good (reviewing) citizen of their academic discipline be a major, minor, or no factor in a tenure decision?
Some commenters said that it should be a major factor, but I think that some of these arguments are based on the assumption that a lousy reviewer will also be a substandard researcher and a bad advisor, in which case, their failure as a reviewer is the least of their problems.
Being a good reviewer is important. Our current system of peer review depends on there being many reviewers willing to spend the time to provide thorough, thoughtful, critical reviews, and to return these reviews in a reasonably timely way. I am not, however, convinced that someone who is a bad and/or disorganized reviewer (turning in reviews late, if at all, for example) is also likely to be a lousy researcher or teacher or advisor. In fact, as an editor, I have had many experiences to the contrary with delinquent reviewers who do outstanding research and who are excellent mentors to their graduate and undergraduate students and postdocs. Some are also excellent teachers, although I don't have this information about most reviewers.
Different institutions have different ratios of expected research : teaching : service by faculty. At my university, reviewing manuscripts and proposals is "service". Service is by far the smallest of the three components, and reviewing is just one part of "service". Perhaps being an excellent (or, at least, not terrible) reviewer is more important at other types of institutions.
Typically, faculty and committees reviewing the files of tenure and promotion candidates at a research institution have a list of journals and funding agencies for which the candidate did reviews, but no indication of the quality of the reviewing efforts. I suppose quality issues could come up in an external letter, but I can't recall having seen any examples of that.
Therefore, at institutions like mine, the quality of someone's work as a reviewer is a non-issue in tenure and promotion decisions unless it is symptomatic of their approach to research and advising, in which case the fact that they are a lousy reviewer is swamped by their deficiencies in these other aspects.Let's assume that we somehow know whether a tenure candidate is a good or bad reviewer. Should it be a factor in employment decisions? I say no: if an individual excels at research and teaching but is a lousy reviewer, I think they should get tenure and/or be promoted based on the fact that they are doing well at the most important aspects of their job.
Being a reviewer shows a commitment to professional service and is also an indication of how well respected and visible someone is in their field. Visibility and respect are important at all career stages, but can be particularly important in promotion to professor at a research university. Even so, there are other indicators of this.
But let's say we still want to know if someone is a good or bad reviewer. One possible indirect indicator of reviewer quality might be the number of reviews someone is asked to do. Those who are lousy reviewers may not get asked to do m/any reviews, and this fact may show up in the numbers, although there are other explanations for minimal reviewing activity.
Can reviewing activity over time be an indicator? Not necessarily. For example, if someone's CV shows that they used to do a lot of reviews and now do not, there are several possible explanations:
- They did such bad or late reviews, or never returned promised reviews, that editors stopped inviting them.
- The topics in which they are most expert used to be well represented in the literature, but now are not.
- Their field of research got more crowded with time, so there is a larger pool of possible reviewers.
- They got too busy with other things and stopped accepting most requests to review.
It may be impossible to tell which of these, if any, is the relevant explanation, so I would hesitate to make inferences from a CV or other document that records a decrease in reviewing activity over time.
Bad reviewers and review-shirkers are annoying, especially since they rely on the reviewing work of others to get their own papers published, but I don't think there should be any major overt punishment of bad reviewers. Although it may seem that review-shirkers are getting away with something, they are losing the opportunity to play a role in the peer review process, to be a constructive influence in their field, and to be respected by editors and others for their reviewing wisdom and efforts. That's their loss, and an appropriate consequence for being a bad reviewer.
13 comments:
I agree that bad reviewers shouldn't be heavily penalised but I wish there was a way to get credit for being a good reviewer. I think I'm a pretty good reviewer - I accept 90% of the reviews I'm asked to do and get them back on time (or sometimes 2 days late). But there is no way to indicate that on my cv.
Well, this explains all the crappy reviews I keep getting (as an author and as a subject editor).
If people think reviewing counts for nothing, then many will not participate or do it poorly. I truly believe the system is broken because of this reality. Question: How can we fix it?
(There are people out there who are responsible and vested, who do constructive and intelligent, on-time reviewing. These folks are completely swamped with review requests, in my experience. How can we increase this pool?)
I don't think the system is broken, although there is always room for improvement of course. There are always going to be diligent people and lazy people, in reviewing as in everything else. Creating a system of punishment to try to get better, more punctual reviewing out of more people is not the way to go because that is not a way to motivate thoughtful, constructive reviews. So maybe the rewards should be greater for excellent, punctual reviews? Some journals do recognize excellent reviewers, but, although most editors make objective decisions about this, I know one who selected all his best friends for the 'award'. No system is going to be perfect. I think we have to find satisfaction in our own hard work and not worry about the slackers (until one of our own papers is a victim of a shoddy review).
I couldn't disagree more. It depends on how bad "bad" is. The scientific enterprise depends critically on everyone contributing to the review of colleagues' and competitors' papers in a fair and timely way. Unfair and late reviews go from annoying behavior to unethical behavior with the degree of unfairness and lateness. Extremely unfair and never returned reviews, I think, crosses into misconduct. It is taking unfair advantage of others for your own gain (since presumably you are continuing to work on the research that is important to you, perhaps even submitting your own papers for review!) Unethical behavior and misconduct should always be considered in criteria for tenure.
First, reviewing for top jounrals in the field is viewed as a sign you are recognized as an important player in your area. I would certainly view it as a plus in a tenure decision.
Second, how in heavens name can I tell who is a "good" or 'lousy" reviewer when reviewing someone for promotion, unless I was the editor on the papers they reviewed. I do thank the gods for good reviewers, and avoid serial lousy ones, but no one will have access to how well a person does at the task.
Finally, being a good scientific citizen has its own intrinsic rewards, some of which are even tangible--keeping up with the latest in your field, and being viewed favorably by leading colleagues.
Mark P
Dear Female Science Professor:
Thanks for your ongoing contributions to the higher education blogosphere.
Thought I'd comment to let you know that you were featured today on Dr. William G. Tierney's blog (co-authored with members of his research center). Here's the link at "21st Century Scholar:" http://21stcenturyscholar.org/2011/02/24/thursday-is-techday-two-blogs-for-the-price-of-one/
I agree with Mark P and the post; go for the intrinsic rewards of being a good citizen (in this case reviewer) and try to instill in early career colleagues the importance of being a conscientious reviewer. This should be a part of mentoring.
Do you think that being a member of the editorial board of a journal gets credit at a tenure or promotion review? One way to get there is to do good reviews and get this recognition for it.
There are too many variables that make punishing scientists for weak peer reviews a very slippery road. Unless there’s a way to determine whether bad reviews are done because of the wrong reasons e.g.: laziness, desire to exploit the situation, plain incompetence, punishments form a very slippery road.
I think the only other time review proficiency should be considered in a tenure promotion is when you have several suitable candidates and they all “score” pretty much the same in all the other areas of scientific proficiency. In practice, it’s not very easy to determine that.
If you don’t mind me sharing my idea (in a nutshell): I envision a new type of centralized peer review model, with standardized peer review criteria/tools and standardized criteria/tools to rate peer reviews by their fellow peer reviewers of the same peer review sessions. We could then quantify this systematically gained assessment of their peer review proficiency i.e. an impact metric for peer reviewing if you will. Using that impact metric, we can create rankings for peer reviewers and specialized “priority conditions” that allow their manuscripts to be more visible (for peer review). So the better you peer review, the better your odds of your own manuscripts being visible, read and peer reviewed themselves.
Technically and functionally very feasible with OA preprint repositories (complimented with special platforms for people to interact and collaborate on) and if it’s open peer review. There are a number of options to provide anonymity too for reviewers, but those are a lot more challenging to accomplish technically/functionally.
I know this isn't quite related to the topic of the thread, but I just had a quick questions. How does one go about becoming a reviewer? I have recently received my PhD and have had the opportunity to do quite a bit of publishing thus far. However, it's not clear to me how one makes oneself available to do reviews. Does one simply have to wait until obtaining a tenure track position? Are postdocs typically given the opportunity to review? If so, is there a way to seek this out? Just curios.
Postdocs can definitely review, and there are a few things you can do to signal your interest in doing some reviews.
You can talk to faculty you know, and perhaps they will pass your name along to editors. For example, when we decline a review request, some journals ask us to suggest alternatives, so if your faculty colleagues knew you were interested, they could start listing you.
You could also introduce yourself to editors at conferences, talk about their journal, and mention that you're interested in getting some reviewer experience. A postdoc did just that with me at a conference a couple of years ago, and I remembered it when seeking a review for a manuscript relevant to his research interests.
If you know any editors personally, you could also contact them and let them know you're interested in reviewing.
Even without being proactive, you may start getting requests anyway. When editors search for reviewers, some search the recent literature or use search functions in the manuscript-handling system that suggests reviewers based on keywords in the literature. If you've been publishing, your name will turn up.
One way that our institution gives credit for reviews is that every faculty member is required to file an annual report with the dean of the school, detailing teaching loads and summarizing evaluations, listing publications and presentations, and detailing service to the department, the community, and the greater academic community. Peer reviewing goes into that last category. Those with exceptional academic service are eligible for reduced departmental service assignments, including committee sabbaticals. Those with insubstantial academic community service are assigned to more onerous institutional service roles. And, in theory, if we ever get pay increases again, good academic community service counts towards merit pay increases.
I'm newish on the tenure track, and the volume of review requests gradually is increasing. In addition to using my publications to identify me as a potential reviewer, I like to think that repeat requests from journals show that editors also rate the quality and timeliness of my reviews highly. Like everyone else I list the journals for which I review on my CV; would it be acceptable/appropriate to put the number of reviews for each journal in parentheses after the journal name? I'm thinking that (for tenure review) I would get the impression that someone is a good reviewer if she/he listed 15 reviewed manuscripts for Journal A versus 2, for example. Thoughts?
Post a Comment