Survey of Impact Criteria in Peer Review

Last week, an e-mail arrived from someone who is

"..part of a research team conducting a National Science Foundation-sponsored study that may be of interest to visitors to your website. It concerns the comparative analysis of the grant proposal peer review processes at six public research funding agencies (http://csid-capr.unt.edu/)."

The current phase of this group's research at the University of North Texas includes an online survey of Impact Criteria in Peer Review. I have been asked to help advertise this survey.

Click here to participate in the survey. It takes about 20 minutes to complete, maybe more if you spend a lot of time pondering the complexities of the questions and possible responses.

After establishing some things about you and your employment status, the survey asks you to indicate which funding agencies are relevant to your life, and in what capacity you have been involved in these agencies.

And then: In your opinion, who should be involved in the peer review process and decision-making? Peer academics, international academics, funding agency officials, other government officials, industry people, "lay" people of the US or other countries, those most affected by your research?

The weirdest part is a series of statements about the philosophy of research and its impact on society. You are supposed to say whether you think your funding agency's "orientation" agrees or disagrees with some statements, e.g. "There ARE NO SIGNIFICANT BENEFITS to society from public funding of scientific/technical research." Strongly agree? No opinion? Strongly disagree? (and some intermediate choices)

Technology happens, society must adapt. Agree.. or not?

And then answer those same questions for what you think the LAY PUBLIC thinks. Is this a measure of how cynical we are?

Another question deals with how we think GOVERNMENT OFFICIALS (and others) value "instrumental" vs. "intrinsic" research.

And yet another: Can the intrinsic value of research be measured? I suppose everything can be measured, but does the measurement mean anything?

That's Part 1, which the survey authors admit is a bit "abstract".

In Part 2, which does not need to be completed, survey responders give their opinions (on a scale of 1-100) about some inventions selected in 2009 by Time Magazine as the most important ones of the previous year. You also guess what the average answer was from "the public" who read the Time article and responded online with their votes. You can see how your guesses correspond to the data by requesting the results via e-mail.

I did not find the survey particularly illuminating, but I wonder what will be done with the results of this survey and the evaluation of the results.

Feel free to leave additional comments here on the FSP blog if you have thoughts and ideas to share about the survey or the issues it seeks to evaluate. Is this a good way to determine what researchers think the public thinks about federally funded research? What should be done with such data?