Tuesday, January 12, 2010

Key Keywords

There's an interesting article in Slate about the possible differences in outcome of scientific research based on the mode of funding: longer-term, more flexible grants vs. shorter-term, project-based grants. The article describes the results of a study that compared Howard Hughes Medical Institute (HHMI) funded investigators with NIH funded investigators who could be considered peers based on publication records and receipt of prestigious awards.

According to the study, the HHMI-funded researchers generated more highly cited work, as well as a lot of publications that had never been cited, highlighting the fact that even overall successful research results in some (many) dead ends. A possible explanation for the success of the HHMI researchers is that they had more freedom to "innovate" (the topic that is the main focus of the article).

That's interesting, but I was particularly intrigued by one of the measures used to evaluate innovation: the use of keywords in a publication:

They [HHMI researchers] are .. more likely to produce research that introduces new words and phrases into their fields of research, as measured by the list of "keywords" they attach to their studies to describe their work.

There's also some tentative evidence that HHMI scholars experiment more than their NIH-funded counterparts: ... their keywords change more often across studies, ... suggesting broader experimentation.

I can see that the variety of keywords for an individual researcher might indicate breadth or interdisciplinarity, but are keywords a good measure of innovation?

I am sure that innovation is difficult to quantify. Even citation indices are not necessarily a good measure. Some very highly cited publications are not innovative but represent a necessary technical advance upon which more innovative research can be based. And there are likely many examples of very innovative work that is not necessarily highly cited (especially if it becomes "conventional wisdom" rather quickly).

If keywords measure something important about a publication, however, perhaps I need to pay more attention to my keyword selection, possibly even making up a few new words now and then. When choosing keywords for my papers, I typically pick a few descriptive terms, but I don't give them a lot of thought. For me, keywords are an afterthought, selected quickly so I can continue with the manuscript submission process. Most of the important words are likely to be in the title and abstract anyway. Clearly I have not been thinking out of the box re. keyword selection.

I suppose if you invent something (even if it's just a new term, like thermofelinics) and it turns out to be important, you might want your paper to appear in searches as the earliest one on record that describes this new thing/process/idea. And for that to happen, perhaps you need to choose the right keywords. Perhaps you need to choose a combination of prosaic keywords and hot new keywords; let's call them: lowkeywords and keykeywords (K2W?), respectively.

So: Do keywords indicate something fundamental about us as researchers? And if so, should I take more interest in keywords and their careful selection?

Keywords: keywords, lokeywords, keykeywords, thermofelinics


Dame Eleanor Hull said...

Thermofelinics! Ooh! I have so much data just lying around the house. Research questions abound. Why are cats hotter when plastered against you in the middle of the night? Why does it take so long for their paws to warm up? Do cat temps vary more with fat or with fluff?

Michael Hultström said...

As for you, key-words are very much an after thought for me. In addition, many journals want key-words that specifically aren't in the title or abstract, so as to broaden the possible search criteria. I think this may skew the results for field where this is the norm.

Research-wise I think key-word characterisation has some merit. Mostly because it is rather un-selected and not something people worry about or try to use to play the system. This may make them less biased than for example title words, which are selected to create the impression of an important finding.

Anonymous said...

I usually ignore keywords as they don't often represent what the articles represent. However, when they do I love them. I guess it's a love-hate situation!

Comrade PhysioProf said...

Keywords? Keywords!? Who's talking about keywords!? Don't talk about keywords! Keywords!?

[Can you guess the reference, FSP? NO ONE TELL HER!]

fubarator said...

I don't like where this is going for those journals where authors have to pick keywords from a LIST!

Anonymous said...


Mark P

Genomic Repairman said...

What drivel key words don't equal good science. Once again utter bullshit from the media. Most of them don't have the attention span to have a decent conversation with us which has been my experience.

Anonymous said...

CPP: FSP is interested in cats and research. I highly doubt FSP has every watched football, let alone post-game press conferences by Jim Mora.

Ms.PhD said...

Okay, totally different viewpoint here. In my field, new keywords and acronyms equate almost directly with money and power. But you have to get people to adopt YOUR name for something. This results in ridiculous arguing over names, even when things are already in the database with a different name. It's a giant pissing contest. It might be funny to watch, if it weren't so bad for science. These kinds of fights have resulted in very fragmented literature, where you can miss entire swaths of work if you don't know what to search for. Makes it difficult for students and newcomers, to say the least.

Most of the people I know who actually do innovative work are not that great at coming up with catchy names. And the converse is also true: those who come up with innovative names, aren't really doing innovative work.

I think the smart ones do use exactly what you describe, FSP: a mix of prosaic and new terminology in their keyword lists. Some people really do put a lot of thought into how their papers will get indexed- which is clever, really. This is essential for grants, too, if you want to steer which study section reviews your work, that's done by keywords.

FWIW, I got my hands on an old HHMI booklet (supposedly for HHMI distribution only, which made me laugh) that included among the instructions for grants and papers how important it is to choose the right keywords and come up with your own terminology to emphasize your innovative contributions. Thus, we can conclude that it is actually part of the HHMI doctrine: thou shalt make thyself look as innovative as possible! [insert evil laugh here]

AcidFlask said...

Cue the inevitable followup metastudy of keywords in papers, which shall itself introduce a new keyword: keywordomics.