Monday, September 12, 2011

A Review's Dilemma

In an editorial published in Nature a few weeks back, Nai-Xing Wang of the Chinese Academy of Sciences argues against the Chinese obsession with publishing their research in journals with high impact factors. In a nutshell, the impact factor of the journal is used by the Chinese government to rate the quality of the research. Wang discusses this at length and makes many good arguments which I will not discuss here (read the editorial, it's worth the time).

I've been aware of this situation for some time, but it has always made me feel uncomfortable for an additional reason - I'm a reviewer for a number of journals published by the Royal Society of Chemistry and am increasingly reviewing more and more papers by Chinese authors. Certainly if the research is of poor quality and should not be published without rework/more data/..., I have no problem in stating so and recommending rejection. But many of the papers are more or less acceptable, but maybe should be published elsewhere, in a journal with a lower impact factor. How do I decide that, or even more importantly, should I be deciding that? A large part of me thinks that my job as a reviewer is to pore over the research and decide if it should be published, but that the publisher and their editors should be the ones deciding if it is appropriate for their journal. The editors all have technical backgrounds so the decision should not be that hard to make.

It could be argued that the reviewers are likely the readers of that same journal and could provide valuable input as to what they want to read in it. I find that argument weak. There are very few journals that I have subscriptions to - I spend most of my time scanning tables of contents from a large number of journals and seldom concern myself with the specific journal that a research article is published in. I will value an article from "Polymer" as much as one from "Soft Chemistry" or "Journal of Applied Polymer Science" or "Macromolecules" - the idea that there is a particularly brand or brand quality is lost to me [1], so I don't feel that I should be in the position of making this judgment for a journal, and now knowing that this decision could have a big impact on the career of the researchers does not help at all.

I'm sure China will eventually change it's ways (seriously, deciding a research article is more valuable because it is published in a journal where the mean impact of previously published articles was high is akin to valuing a single company's stock based on how well the Dow Jones did over the last 2 years!), but that doesn't make my job any easier in the present.

[1] Now if the article is published in the "Major Asia Minor Journal of Polymers, [2] I might not be quite as interested, or at least I would be initially more skeptical of the data, results and conclusions.

[2] I've have nothing against researchers from Asia Minor or it's present day descendants - the name Asia Minor has always hit me as funny ever since I was a child.

4 comments:

Anonymous said...

Editors should be the ones making the decisions, not the referees. At Nature Chemistry we basically ask refs what they think of the work - the good points and the bad points. Is it technically sound? Is literature cited appropriately and so on. Sure, referees can inform us of prior art, but ultimately, the editors decide if the manuscript under consideration is enough of a step forward for our journal. We wrote an editorial on it back in Jan 2010. See: http://www.nature.com/nchem/journal/v2/n1/full/nchem.488.html (free nature.com registration required to read it).

John said...

Stu,

Thanks for the input. The editorial was great to read, especially seeing the feedback to the reviewers.

I don't mind separating chaff from wheat, (or telling someone to keep grinding) but I don't think that I am best qualified at telling wheat from rye from maize (figuratively of course).

Although by the way, I couldn't get the link you provided to work. I think this one might work.

Anonymous said...

John - thanks, and thanks for posting a link that works, sorry about that!

Anonymous said...

Hi there, just required you to know I he added your website to my Google bookmarks due to your layout. But seriously, I think your web web site has 1 in the freshest theme I??ve came across. It extremely helps make studying your website significantly easier.