I’m a bit intrigued by the peer-review publishing process so acutely familiar to those in the scientific and medical professions. Over the past few months I’ve had conversations with doctors, researchers, and scientists about their personal frustrations with the “tyranny of peer review”. Having never attempted to publish something myself, I confess to knowing very little about the whole process.
I was, however, inspired by Beatrice Golomb’s Edge Article where she lays out an argument for reforming the system. I share her optimism. Given the rate of change in the world as the smartest people self-organize online, it seems hard to imagine that the current institutional structures can continue much longer. In Spoonful of Medicine: Nature Medicine 2.0, and the editorial it references, the Nature editors ask us to consider what “the Web 2.0–driven scientific publishing world going to look like?” and whether Nature will survive.
That they chose to discuss the topic from this perspective makes it easy to answer: No, it will not.
Ok, scratch that. Nature has a much better shot of being around after the world forgets about del.icio.us; it just won’t be as relevant as it is today.
How on earth can I confidently declare Nature’s demise? They are not asking the right question!
I totally agree with Attila Csordas’ point that this isn’t a winner-take-all scenario. The issue is ongoing relevance. The real question is, “How can we use these emerging tools to create an even better journal for our readers?”
Based on my recent conversations, here are the top-10 problems with the current system. (Please note that these are comments about the entire process and not directed explicitly at Nature).
- Unstated real or perceived conflicts of interest. Reviewers and authors can have relationships with entities that have an ulterior motive in getting material published.
- Peer-review process advances slower than scientific progress.
- The current process does not provide authors and reviewers with basic collaborative web tools.
- Authors lose copyright privileges when publishing yet are often forced to publish to continue career advancement.
- Peer-review networks tend to form around cliques. Those “outside the club” of a particular discipline – where often the best ideas surface – cannot get published because new ideas are rejected by the current establishment. As a result great ideas are often lost.
- Precedence is often establish by those with the best personal contacts and not those who first introduce new theories.
- There is no medium for wider, instant dissemination. Doctors or researchers who prepare a presentation or speech cannot “publish it” to a wider audience.
- Participating in the review process has little benefit for the reviewer. Performing reviews can take an enormous amount of time and the written reviews are not themselves “published”.
- Reviews and reviewers are not “reviewed”. An author who receives a biased review or one based on poor critical thinking has no recourse to publicly respond or invite others to comment.
- Journals can be prohibitively expensive for some in the developing world.
As I stated from the outset, I’m remarking on something which I know little about. I do, however, know how to ask questions, listen, and type. And this is really all Nature needs to do to take the first steps in the pursuit of ongoing relevancy.
{ 1 comment… read it below or add one }
Hi Kevin,
I’m an editor for BioMed Central, an open access biomedical publisher, so I might be able to resolve some questions and misconceptions in your post about peer review.
—
1. Unstated real or perceived conflicts of interest. Reviewers and authors can have relationships with entities that have an ulterior motive in getting material published.
True, but many journals, such as mine, require authors and reviewers to declare their competing interests – in our medical journals, these interests are published with the article. Editors are used to watching out for this.
—
2. Peer-review process advances slower than scientific progress.
Yes, but peer review doesn’t stop someone first posting their article on their own web-site, discussing their work at conferences, or posting their work on a pre-print server like ArXiv. Anyway, scientific progress isn’t as rapid as people believe, and without the checks and balances that peer review gives, all sorts of rubbish would be published, and scientists would have to follow even more blind alleys than they do already after reading profoundly flawed research. Peer review adds some rigour into the process of communicating scientific research. Less haste, more speed is an apt concept here.
—
3. The current process does not provide authors and reviewers with basic collaborative web tools.
That’s nothing to do with peer review, just the delays in the Web 2.0 revolution getting to publishers. PLoS ONE (published by Public Library of Science, another OA publisher) does now offer reviewers and authors interactive tools to annotate articles. Many journals, like mine and the BMJ, allow any reader to comment on a published article.
—
4. Authors lose copyright privileges when publishing yet are often forced to publish to continue career advancement.
Traditional journals insist on copyright transfer. Many open access journals, including those published by BioMed Central and PLoS, allow the authors to retain copyright. The article is published under a Creative Commons Attribution License.
—
5. Peer-review networks tend to form around cliques. Those “outside the club” of a particular discipline – where often the best ideas surface – cannot get published because new ideas are rejected by the current establishment. As a result great ideas are often lost.
I don’t believe that this complaint is really that valid. The complaints I’ve read about were by top scientists who couldn’t get their idea published in Nature, Cell or Science. Well, just publish it elsewhere. There are plenty of journals that aren’t as picky as those journals, and if authors had a little more self-awareness they’d recognise that they aim too high sometimes. Besides, many journals don’t use established lists of reviewers, but go straight to those publishing related work and ask them. So, yes, you usually have to be a published scientist to review, but then it is called *peer* review, isn’t it? I doubt that “the best ideas” surface outside academic research, the lone researcher is more likely to be a kook than a genius. There are some geniuses out there, but they are the ones you read about in the news – there’s a teensy bit of a selection bias going on…
—
6. Precedence is often establish by those with the best personal contacts and not those who first introduce new theories.
I don’t see the basis for this argument. Precedence does go to those who first raised a theory, so long as scientists are aware of it. Those who publish in languages other than English are at a disadvantage, admittedly, but some journals allow republication of work in English that was previously published elsewhere in another language, so that gives authors the possibility to widen their audience. Peer reviewers go out of their way to alert authors to work that first demonstrated something, and I have also insisted that authors cite certain studies. Scientists are very attuned to giving due credit for the origin of ideas or techniques.
—
7. There is no medium for wider, instant dissemination. Doctors or researchers who prepare a presentation or speech cannot “publish it” to a wider audience.
Yes, they can. ArXiv and other pre-print servers allow the publication of non-reviewed work (see e.g. http://pkp.sfu.ca/). Theses and dissertations can be published electronically (e.g. http://www.ndltd.org/browse, MIT on DSpace). This Portugese university repository, for example, allows the publication of reports, presentation etc. (http://papadocs.dsi.uminho.pt/). If a university doesn’t have a repository for this kind of material, then it should do! Staff and students can take the lead, rather than waiting for journals to do it for them – journals are traditionally for peer-reviewed research, why would we necessarily expect them to post presentations? That said, Nature has recently launched Nature Protocols, so publishers are making some effort to include material that is outside their usual range.
8. Participating in the review process has little benefit for the reviewer. Performing reviews can take an enormous amount of time and the written reviews are not themselves “published”.
Reviewing takes between 2-6 hours, according a survey I read. I’ve seen reviews done in 10 minutes…
Here are a few reasons for participating in peer review:
- Allows a researcher control over what is published in their field – they are the “gatekeepers of science”.
- Allows researchers to ensure that what is published accurately reflects and acknowledges their field.
- Can help a scientist get promoted and get grants, as journals often list the names of their reviewers annually.
- In the case of the medical journals in the BMC-series, published by BioMed Central, we *do* publish the reports, along with the name of the reviewer.
- Reviewers are actually paid by a small minority of journals, and more commonly can get other perks such as discounts on reading or publishing in the journal.
- Reviewers get the opportunity to read their competitors’ work months before it will be published, and unscrupulous reviewers can deliberately block publication.
- It’s interesting! They’re scientists, they enjoy critiquing science!
9. Reviews and reviewers are not “reviewed”. An author who receives a biased review or one based on poor critical thinking has no recourse to publicly respond or invite others to comment.
Not true. Editors assess the reviewer reports and qualifications. Authors who receive what they perceive to be a biased review can appeal to the editor, and request a further opinion. If they are badly treated and the journal is a member of the Committee on Publication Ethics (such as BioMed Central, BMJ, Lancet) then authors can even take a case to that body. Some journals (BMJ, Lancet) have an ombudsman.
10. Journals can be prohibitively expensive for some in the developing world.
Yes – this is one of the reasons why open access is a good idea! The research is free to read for anyone with Internet access. Traditional pay-to-view journals are also members of a scheme called HINARI, a WHO project that allows some people in developing countries to read the research for free (but it does have limitations, as they need to be connected to an institution).
—
It’s easy to criticise peer review, but it’s harder to come up with a better system.
Regards,
Matt