Wednesday, December 19, 2007

Not Rated

The subject of rankings of colleges/universities, departments, and programs is huge and complex and I am not even going to begin to tackle it in a comprehensive way. I am, however, interested in the topic of how administrators decide which rankings they like and don’t like.

You might think it is simple: administrators like high rankings and don’t like low rankings, but it is actually more complex than that when you consider the relative rankings of departments within a university or unit of a university.

For example, there is one particular science/engineering department at my university that is typically very highly ranked in major surveys of graduate programs. I think it is in fact a very excellent department. I have worked with faculty and students in that department, and served on university committees that evaluate faculty and students in that department, and there is no question that it has many strengths. Owing to its excellence, this department receives a lot of attention and resources from the university, and, to be fair, generates a lot of attention and resources for the university. Administrators love this department.

Consider now the situation in which another department (just for fun, let’s say this is My Department) ranks extremely high in a recent survey, but the Esteemed Department does not rank as high. Are the Deans et al. happy that My Department is highly ranked? No, they are not happy. In fact, because the Esteemed Department is not as highly ranked as they expected it to be, the survey must be flawed. In fact, the Deans et al. are so disturbed by the unexpectedly low ranking of the Esteemed Department (i.e., not even in the Top 10), they take no joy whatsoever from the high ranking of any other department and bury the data.

If the Esteemed Department had been ranked #1 or #2, the media engine would be roaring and we would all be reading glossy university magazines and brochures that heralded to the world the amazing success of this department. Are the people in My Department angry and bitter? No, they are not angry and bitter. You might say, however, that they are chagrined. They are also considering options for self-promotion that show that they are proud of their accomplishments but at the same time are not slaves to rankings, which we all know are deeply flawed, or at least, some of them are.

5 comments:

  1. X-Rated

    Touchy subject...

    At MyU there is a similar situation. When resources are heavily directed to one department, the question often arises: "Would we be better off with one very strong (~highly rated) department and leave the rest comparatively weaker or would we be better off spreading resources around?"

    Unfortunately, when an administration lays the money down on a horse and that horse is outperformed, they look bad.
    Can't have that?

    At to ratings, they are a necessary evil, and the more transparent they are - do not include points for dead Nobel prize winners - the better. Such ratings can be used to demonstrate bs on the part of administrators.

    At MyPlace we have the "ambitious aspiration to be one of the top three public research universities [in the world]." Which would be nice, except we don't have the resources. The cumulative effect of many ratings out there is to make it clear that this goal is unrealistic and inappropriate for our institution.

    Ratings also help those not privy to the inside dope to have some information about the relative strength of universities and departments. Used cautiously and appropriately they can be valuable.

    ReplyDelete
  2. As far as I can tell, from the administrator perspective, the only ranking that actually matters is how much money a department brings in for the school (both through grants and donations). The benefits of having a highly ranked department in an area with high grant funding opportunities means a higher potential for more income. It also sometimes aligns with press articles that lead to big donations.

    More people will donate money to quality, but medium ranked departments that are trying to cure sick children than very highly ranked departments in less directly practical topics.

    My assumption is based on a very small n so I'm curious if this is true in your case. Would press releases etc in the medium ranked department give the school a bigger bang for the buck in general?

    ReplyDelete
  3. At the very least, your department should trumpet the favorable ranking on your departmental webpage and in information you send out to prospective grad students, alums, etc. That audience will be more interested in comparing you to competing departments within your field in other universities; they won't be interested in how you stack up against some other department in your university. No single set of rankings is perfect, but to do well on one presumably is meaningful along some dimension and your department should get whatever mileage out of it that you can.

    ReplyDelete
  4. nice use of font size...very creative...

    ReplyDelete
  5. The rankings that really count are the NRC surveys made each decade. They are gearing up right now, so it may be too late.

    ReplyDelete