Review Well or Die

In the course of some recent discussions about Journal Editing, the question arose as to whether someone's reviewing skills should be a factor in tenure and promotion decisions. In particular, should a tenure-track professor's failure to be a good (reviewing) citizen of their academic discipline be a major, minor, or no factor in a tenure decision?

Some commenters said that it should be a major factor, but I think that some of these arguments are based on the assumption that a lousy reviewer will also be a substandard researcher and a bad advisor, in which case, their failure as a reviewer is the least of their problems.

Being a good reviewer is important. Our current system of peer review depends on there being many reviewers willing to spend the time to provide thorough, thoughtful, critical reviews, and to return these reviews in a reasonably timely way. I am not, however, convinced that someone who is a bad and/or disorganized reviewer (turning in reviews late, if at all, for example) is also likely to be a lousy researcher or teacher or advisor. In fact, as an editor, I have had many experiences to the contrary with delinquent reviewers who do outstanding research and who are excellent mentors to their graduate and undergraduate students and postdocs. Some are also excellent teachers, although I don't have this information about most reviewers.

Different institutions have different ratios of expected research : teaching : service by faculty. At my university, reviewing manuscripts and proposals is "service". Service is by far the smallest of the three components, and reviewing is just one part of "service". Perhaps being an excellent (or, at least, not terrible) reviewer is more important at other types of institutions.

Typically, faculty and committees reviewing the files of tenure and promotion candidates at a research institution have a list of journals and funding agencies for which the candidate did reviews, but no indication of the quality of the reviewing efforts. I suppose quality issues could come up in an external letter, but I can't recall having seen any examples of that.

Therefore, at institutions like mine, the quality of someone's work as a reviewer is a non-issue in tenure and promotion decisions unless it is symptomatic of their approach to research and advising, in which case the fact that they are a lousy reviewer is swamped by their deficiencies in these other aspects.

Let's assume that we somehow know whether a tenure candidate is a good or bad reviewer. Should it be a factor in employment decisions? I say no: if an individual excels at research and teaching but is a lousy reviewer, I think they should get tenure and/or be promoted based on the fact that they are doing well at the most important aspects of their job.

Being a reviewer shows a commitment to professional service and is also an indication of how well respected and visible someone is in their field. Visibility and respect are important at all career stages, but can be particularly important in promotion to professor at a research university. Even so, there are other indicators of this.

But let's say we still want to know if someone is a good or bad reviewer. One possible indirect indicator of reviewer quality might be the number of reviews someone is asked to do. Those who are lousy reviewers may not get asked to do m/any reviews, and this fact may show up in the numbers, although there are other explanations for minimal reviewing activity.

Can reviewing activity over time be an indicator? Not necessarily. For example, if someone's CV shows that they used to do a lot of reviews and now do not, there are several possible explanations:

- They did such bad or late reviews, or never returned promised reviews, that editors stopped inviting them.

- The topics in which they are most expert used to be well represented in the literature, but now are not.

- Their field of research got more crowded with time, so there is a larger pool of possible reviewers.

- They got too busy with other things and stopped accepting most requests to review.

It may be impossible to tell which of these, if any, is the relevant explanation, so I would hesitate to make inferences from a CV or other document that records a decrease in reviewing activity over time.

Bad reviewers and review-shirkers are annoying, especially since they rely on the reviewing work of others to get their own papers published, but I don't think there should be any major overt punishment of bad reviewers. Although it may seem that review-shirkers are getting away with something, they are losing the opportunity to play a role in the peer review process, to be a constructive influence in their field, and to be respected by editors and others for their reviewing wisdom and efforts. That's their loss, and an appropriate consequence for being a bad reviewer.