If you have published at least one paper in a journal that is included in a citation index, how often do you check your own citation statistics? Do you know what your h-index is? (I mean, do you know the number, not what the h-index is.) Do you subscribe to a citation alert service so you are notified when someone cites your paper? Do you check up on the citation data of certain colleagues, whether or not your intentions are good?
I am not so interested in opinions about whether you think citation indices are a nifty metric or the worst thing to happen to academe since faculty meetings were invented, but go ahead and rant in the comments if you must.
Mostly I am wondering how often we are checking on our own citation data, no matter what we think of the significance (or lack thereof) of these data.
14 years ago
33 comments:
I try to check my citation statistics a few times per year. Although these metrics may not be perfect, they can provide a useful information.
I think it's important to keep track of how my citation rate & H-index are changing over time (is my science impact improving?).
I also use these statistics to see which papers are being cited a lot and which ones are not getting much interest. I can also assess how widely my papers are being cited (by scientists in related disciplines). This is valuable information to help one plan future research and future papers.
I also occasionally check the citation stats of other PIs in my science center. The reason is that my superiors tend to just count pubs, rather than look at their impact. Knowing that my citation rate is one of the highest in my center is important when I negotiate for raises/promotions, additional lab space, equipment, or other resources. Armed with that information, I can more readily defend the importance and quality of my work (along with other evidence, of course).
I usually include a chart in my performance evaluation package showing how my citation rate has increased over the previous year (or whatever the evaluation interval is). I sometimes do separate charts to show how key papers are being cited.
I've written a few other thoughts about citation statistics at this site: (http://womeninwetlands.blogspot.com/2009/05/from-publish-or-perish-to-whats-your-h.html)
I have no idea what my h-index is, and I don't sign up for citation alerts (I didn't even know there is such a thing). I check the number of my citations only when there is a very specific reason to do so - such as when I am asking for a salary raise, or to be converted from a contract researcher to permanent staff, and therefore when I need as many quantitative metrics to back up my case for why I should be given a raise or promotion. (incidentally this has yet to work, i have not received a raise or promotion/conversion in years despite having a higher number of publications AND citations than most of my colleagues and boss. My boss alwasy says that my productivity is very impressive and better than that of most of the permenant staff in the department. Yet my requests for promotion/conversion or for raises are always ultimately denied. Good ol' glass ceiling again I suppose)
It never occurred to me to look up my citation statistics, but I do have a subscription to a citation alert service so that I find out if anyone cites a paper for which I'm an author.
I don't really care how I compare to the rest of the world, but I strongly suspect that an article which cited work I participated in would be on a topic of interest to me...
My current institution is "concerned" about our H-index, so I check on it anytime I have to submit my CV for internal paperwork.
I don't actually pull my own citation statistics more than once a year (usually when I'm writing my Yearly Accomplishments Review), but all of my papers are set up on our library's citation alert service, so I know when any of them have been cited. There's an element of vanity to this ("Yay! More citations! Go me!"), but it is an easy way to keep up on recent literature that is directly relevant to my research.
The funny thing is that before reading your post I had no idea an h-index existed. I'm in the Humanities, ao we are not as aware of these new developments.
So I installed an h-index counter on my computer as soon as I read your post. Now I'm checking the numbers of every single person I know. This is so much fun. :-) Thank you for the great suggestion.
I'm sure you're right, there's going to be lots of discussion of the pros and cons of the h-index etc.
I check my citation stats weekly just because I'm geeky about data, and Web of Science updates their database every Friday. Citations are data! I keep track of how many citations every paper I wrote has, and I know what my h-index was every year of my career going back to paper #1. It is totally all about having data to play with, data that even has something to do with me!
Yeah, in some sense the data have a meaning related to "how well am I doing" and I'm curious about that. But mostly I don't think about my research career progress in terms of h-index and citations. It's really all about the data.
(And yes, there's a side benefit of coming across interesting papers that have cited mine. That is certainly a benefit but not the main reason that I check my citations weekly.)
Clarissa,
Installed an H-index counter? Do tell.
I have to use Web of Science, which offers an h-index output, but can get confused with common names. Is there an easier way?
I run a citations, papers, and h-factor for the faculty in my department yearly and track changes, just to see what's hot. It's also a useful tool to check out people for letters of recommendation, nominations for awards, or general curiosity.
I think your poll leaves out an important factor. I, for one, check my statistics quite regularly but mainly because I am an early career person and need to make sure that some one is reading my work (if only the abstract). On the other hand, I had been in the status where you are currently, I think I would care much less of how many citations I do or do not have.
Finally, I have no opinion on the h-Index because it is not a criteria of interest in the engineering field as far as I know.
Hi FSP!
Just a follow up on my earlier comment on h-index, I looked up more information on it and I think that it CAN become a good criterion for engineering people since it does account for conference papers (which are refereed in most cases). I wonder though how does h-index deal with self-citations. Any information on this?
Also, I just checked my own h-index by downloading a firefox add-on for Google Scholar and my h-index is anywhere between 8 and 11. Do you know what IS a good h-index? Finally, what is the best tool and website to use for h-index calculation?
It would be great if you (or some one else) could respond to these queries for a junior academic :)
So, how do we check our citation statistics? Shy of looking up all my papers on ADS and seeing how many people have cited them, I have no idea how to do the things you mention here. H-index? Citation alerts? How does one find these things?
I check my H-index and related statistics regularly, but only because I know this is a measure that will be implemented by people who _have no idea about the science I am doing_ - ie the people who make decisions on promotions and other pertinent job-related issues (although I agree that citations is a good way to check who is looking at your work, and who's work may be relevant that you may not have otherwise noticed).
It is likely it will be utilised extensively in the upcoming RAE (UK research assessment exercise), and is finding favour primarily amongst people with a high index (older faculty, as it is cumulative, and methods people, who tend toward high cite counts), as well as bureaucrats who like to have an easy one-fits-all measure (which this is not, but appears as).
A good summary can be found at doi:10.1021/cb9001014 for those who are less familiar with this measure.
John V: Firefox has a h-index calculator add-on, I suppose this is what Clarissa meant above? It is based on Google Scholar and linked from wikipedia (along with some other calculators):
http://en.wikipedia.org/wiki/H-index#Computing_the_h-index
I just installed it and tested it vs. the Scopus and WebofScience h-indices. Interestingly, values are really different (Google Scholar index is 4x larger than the Scopus one, which is twice the WoS one), probably because of different database coverage.
for the citations, it strongly depends on my mood. When I'm in a good mood (== feeling I'm making progress) I never check. When I'm in a bad mood I check every day, hoping for the small chance that the number has changed and it'll cheer me up...
what's the h-index? I'll have to look that up
The original paper proposing the "h-index" as a useful metric can be found at http://www.slac.stanford.edu/spires/find/hep/www?bb=physics/0508025.
And yes, particle physicist write papers on evaluating how people are doing in particle physics.
http://www.slac.stanford.edu/spires/find/hep?c=PNASA,46,16569 lists some papers that cite the above, and some of them discuss some possible meanings and deficiencies of the index.
I know mine, and check my citations a few times a year.
I only checked the citation index (intensely) when applying for a green card. Haven't really been interested, since. But then, I am a staff member and not a PI.
Pika,
Found it, works in Safari, too. It's really a Google Scholar gadget. Very easy to use, but not as good as Web of Science on first try. It missed some, had no instructions, and glommed two of my papers together because the titles were similar, and counted only 90% as many citations.
I definitely subscribe to citation alerts. I also used to tell my advisor and coauthor everytime I received a new alert. (I stopped when they didn't seem as excited about it as me.) It's very gratifying to be cited; it means someone else cares about the work and it's actually being used to push science forward! Woo-hoo! It's so easy to feel isolated and like no one cares about your work or like your work doesn't matter. I suppose if/when I have lots more publications and citations, I won't feel that way?
Just found an interesting position of one of the Australian research councils re metrics:
http://bjoern.brembs.net/news.php?item.601.11
ie, that they are 'unscholarly' and 'unfair'
I track my h-index fairly often (about once a month). There are various tools - "publish or perish" is a little windows program that works based on google scholar, but is liable to over-count. If you have a common name, ResearcherID is best - you set it up and manually tell it what all your papers are, but then you (or anyone) can go to your ResearcherID page to get up-to-date Web of Science counts. Scopus has a similar system (which for me gives a slightly higher h-index). Again they try to give every author a unique ID to link papers together, but if they have got it wrong you have to email them to ask for a correction.
I think it is well worth keeping up with these things and checking the accuracy of the tools out there. Partly to find other interesting papers, and partly because I think evaluation by h-index will become increasingly common, so you might as well make it work for you.
I subscribe to a citation alert service to be notified when someone cites some of my papers, but mostly because I want to know who is reading my papers and how my findings are being used. This helps me also advertise as much as I can papers that are not getting to many citations but I feel are important.
I know my H-index with a +- 1 uncertainty, mostly because my husband (who likes to gossip about H-indexes of other people) keeps me informed.
I'd just like to complain that I can't vote in the poll (probably) because my wife voted this morning from the same IP address. Another aspect of the two-body problem that no one considers! [shakes fist at sky]
And yes I know my H index, and I know which of my papers are undercited and which are overcited. I guess some papers just need more time to reach their potential (like children).
I prefer Hbar (H/yrs since PhD) or total citations as far as meaningless metrics go.
"It's really a Google Scholar gadget. Very easy to use, but not as good as Web of Science on first try. It missed some, had no instructions, and glommed two of my papers together because the titles were similar, and counted only 90% as many citations."
I had the opposite experience---Google Scholar listed some of my papers twice, mixed in a few by other authors with the same last name, and got so many citations for others that it doubled the h-index Web of Science computed. I could clean up the "other authors" problem, but the numbers were still a bit on the high side (though I think the Web of Science numbers are a bit low, so I don't know which is better).
"If you have a common name, ResearcherID is best - you set it up and manually tell it what all your papers are,"
I set up a ResearcherID, but I found it very difficult to maintain the list of papers (one of the most awkward web interfaces I've dealt with) and I stopped trying to maintain it. It was a good idea, ruined by a lousy implementation.
I've started checking citations about once a year to keep my CV up-to-date for promotion+tenure committees. I didn't check when I was a grad student, and once I'm a full professor I'm not going to check anymore. It makes me anxious. Metrics in general are bad for me---any time I start paying attention to a metric, I start becoming obsessed with trying to optimize it. And I have a feeling h-index is highly optimizable if you're willing to throw your reputation to the wind.
h-index gives a good first order approximation to research quality, but from a sample of 50+ people about one in 10 people have an h-index that is much higher or lower than their true standing. Using a metric that is wrong one in ten times for individual performance measurement sounds like an invitation to be sued.
How about p-index?
http://www.springerlink.com/content/057163k612473667/
I think regularly checking one's h-index and citation numbers is a sign of underlying insecurity.
Kris,
your link refers to use of journal rankings, not rankings of individual papers or people.
Kevin,
our difference might arise from my having an uncommon name, which allows more accurate results from a simple search.
anon@8:49
a 90% accuracy seems good enough for inclusion in a balanced discussion. What measure of performance for a scientist exceeds 90%?
I check mine in a somewhat obsessive manner right now. I am preparing for my promotion to full and my chair is obsessed with citations. It's probably a bit neurotic on my part. I have the metrics for recent promotions to it makes me feel better, although I really just need more citations and it is mainly time that will help there.
"Kevin,
our difference might arise from my having an uncommon name, which allows more accurate results from a simple search.
"
Actually, my last name is pretty rare (I think there are 50-60 of us alive with this name). Unfortunately, many of them are scientists, and two are even in very close subfields to mine, so I have to be careful to clean the search.
Web of Science gives me an h-index of 21, but is missing some important publications (like patents and refereed conferences). Google gives me 30, after I do an advanced search with initial and last name: author:"f lastname"
At least one paper appears twice, though, so the Google Scholar results may be +- a couple (depending whether split papers count twice aove the h-threshold or end up below the threshold when the combined count would be above threshold).
We have to report our citations and h-index annually as part of the merit review process in our department.
I find tracking my citations is also useful for finding useful papers in my field - i.e. those that cited me - and is useful for understanding which lines of research are of more or less interest to the science community.
Post a Comment