Tuesday, August 17, 2010

Eliminating Supplemental Information

The Scholarly Kitchen is reporting on the Journal of Neuroscience discontinuing "Supplemental Information" (often called "Supporting Information"), abbreviated SI. This is information that is freely available on line to help support a published article. In chemistry, this can often include NMR or FTIR data to show that the molecule that was stated to be synthesized was in fact synthesized, or it can be additional plots run with different samples to show that the representative plot in the article was in fact representative. (This happens a lot in rheology as rheologist often have a great excess of data.)

I've always found the information (or at least knowing that it exists) to be helpful. By having SI, articles can be kept shorter and have good flow - the details are elsewhere, much like a footnote. Articles often have page restrictions, but SI, being electronic, is without such restrictions.

The editors of the noted journal sees all of this as problematic for both the authors and reviewers.

For the authors,
"With few restrictions on space, reviewers may place additional demands on authors, requiring them to perform and add new analyses and experiments to the supplemental data. Often these additions are “invariably subordinate or tangential,” Maunsell maintains, but represent significant work from the author and thus delay the publication process. Supplemental data thus changes the expectations of both author and reviewer, leading to what he describes as an “arms race:”
And for the reviewers,
"Reviewer demands in turn have encouraged authors to respond in a supplemental material arms race. Many authors feel that reviewers have become so demanding they cannot afford to pass up the opportunity to insert any supplemental material that might help immunize them against reviewers’ concerns."
But there is also the truly laughable perspective of the journal:
"Validating supplementary data adds to the already overburdened job of the reviewer, Maunsell writes. Consequently, these materials do not receive the same degree of rigorous review, if any at all. At the same time, the journal certifies that they have been peer-reviewed."(emphasis added)
I say laughable as while I greatly value peer-review and am an active participant and strongly believe it should continue, I never look to a journal to certify anything. A peer-reviewed article has simply been sent out to reviewer and they have sent reviews back. I don't know what the reviews were, if they were done by competent people, if they loved the article, hated it or whatever, and I also don't know if the reviews influenced the editor in the least. I certainly know that everyone reviews articles in their own manner and that that manner may not even be consistent from paper to paper. (I'm not.)

Add to this the knowledge that any paper can later be retracted and you have reduced the supposed certification to vapor. Peer review: three people looked at it and liked it. That's all, nothing more. It is not a proof of correctness, it's just proof that three people looked at it and (probably) liked it.

Tip of the hat to Matteo Cavalleri for the lead.

1 comment:

孙尉翔 said...

The concept that peer-reviewing ends at the publish of a paper is misleading. In fact the paper continued to be peer-reviewed.

However, I favor the concept that peer-reviewing is a big burden. That, to save more peoples time, is why people have the manuscript first reviewed by a small group of peers before it appears in front of a vaster audience.

Error in the first understanding can lead to error in the second, though, as the editors of the journal had in their annoucement.