Faculty of 1000

Post-publication peer review

Archive for the ‘Literature’ Category

Faculty of a Million?

Posted by rpg on 1 April, 2010

Apparently, the two self-proclaimed ‘top’ scientific journals, Nature and Science, have ended their hundreds of years-old feud and teamed up to launch a new journal, to be called either Scientific Nature or Natural Science, depending on the result of a text-message vote by the scientific community.

Sounds good? Well, not really. We’re a bit upset that they’re also creating a social networking site called ‘Faculty of a Million’, funded by a grant from Facebook, where scientists can vote papers for acceptance by pressing a “Like” thumbs-up button or reject the paper by pressing a “Dislike” button. Our company lawyer has just had an apoplectic fit.

And this seems to be a direct invasion of privacy:

readers will have the option of Skyping authors directly to share their thoughts and feelings about a paper simply by clicking that author’s name. As an added incentive, the first 100 new subscribers will get free genome scans.

It’s a nice cover though:
Easter bunny

Posted in f1000, Friday afternoon, Literature, The Scientist | Comments Off on Faculty of a Million?

Publish or perish – a question of ethics

Posted by Callum Anderson on 15 March, 2010

I got a very strong sense of deja vu when leafing through PLoS Biol recently. I was sure I had seen something very similar to Jeffrey Shaman’s paper Absolute Humidity and the Seasonal Onset of Influenza in the Continental United States before.

A quick check on PubMed proved me right. I found the following, published two months earlier, in PLoS Curr Influenz:

Absolute Humidity and the Seasonal Onset of Influenza in the Continental US
Jeffrey Shaman,* Virginia Pitzer,† Cecile Viboud,‡ Marc Lipsitch,§ and Bryan Grenfell

PubMed ID 20066155

Because this was PLoS, I was also able to print the full paper and compare. I couldn’t find any differences whatsoever between the two papers. In fact they were exactly the same except for a reshuffling of author order and an abbreviation in the title.

A quick check back on PLoS Biol and I notice that someone else has seen the discrepancy. A comment attached to the article begins with the following

Compare, published in PLoS Currents influenza (dec 18th)
Absolute Humidity and the Seasonal Onset of Influenza in the Continental US
Jeffrey Shaman,* Virginia Pitzer,† Cecile Viboud,‡ Marc Lipsitch,§ and Bryan Grenfell

PubMed ID 20066155

with (and not cited, if I am not mistaken)

Absolute Humidity and the Seasonal Onset of Influenza in the Continental United States (23 february 2010)

Jeffrey Shaman1*, Virginia E. Pitzer2,3,4, Cécile Viboud2, Bryan T. Grenfell2,4,5, Marc Lipsitch6,7,8

When this poster commented, only one of the articles was listed in PubMed. A search for “Absolute humidity” on PubMed today however yielded the following results [click it to get full size]

A PLoS spokesperson had answered the comment in less than 3 hours (perhaps they anticipated something being said). Their official line was as follows

PLoS Biology is fully aware of the authors’ submission to PLoS Currents referenced above. PLoS Currents is a website for immediate, open communication and discussion of new scientific data, analyses, and ideas in a critical research area. The work is screened by experts, but is not subject to in-depth peer review…

Our policy until now (February, 2010) has been to allow resubmission of PLoS Currents content to another PLoS journal. However, the decision to include Currents in PubMed (and PubMed Central) has caused us to reconsider the status of content communicated via Currents, relative to other journals.

I am certainly not convinced by this argument. Having personal experience of getting journals into PubMed, it is not something that happens immediately; the typical process is eight to twelve weeks and PLoS Curr Influenz was already accepted by PubMed in 2009. The accepted date on the re-submitted paper in PLoS Biol was January 20, 2010.

And even worse still, the received date of the paper by PLoS Biol was September 10, 2009. PLoS Curr Influenz did not even accept the duplicate paper until December 18, 2009.

The dates simply don’t add up, a journal doesn’t just email PubMed and expect to show content the next day, and feigning innocence just makes PLoS look at worst deceitful and at very best incompetent. If PLoS was aware that the paper had been submitted to both journals, and was aware that PLoS Curr Influenz would be listed on PubMed, they should have made a full disclosure on the paper subsequently published in PLoS Biol.

Now, I am very much in favour of rapid communication journals, I think they represent an excellent platform to publish cutting edge research, but a distinction between these and traditionally peer-reviewed journals must be drawn somewhere. Should a publication like this really be submitting content to PubMed when their editorial policy allows re-submission in other PLoS journals? PLoS have been having their cake and eating it for a long time now. In a world where publication stats are frequently used as a method of judging the worth of a researcher, are the authors here benefiting twice from the same paper? And PubMed has a very clear policy on duplicate articles, which PLoS should know about.

So why didn’t they do it? Why didn’t they tell PubMed that they would be knowingly supplying duplicate articles? Well I do have a theory [snip-snip – F1000 Lawyers]… But it would be much better to see what you think.

Posted in Journalism, Literature | Tagged: , , , , , , , , , | 24 Comments »

Food for thought

Posted by Callum Anderson on 17 February, 2010

A recent evaluation on Faculty of 1000 Biology highlights a novel advance in the fight against adolescent obesity.  In what could be considered the first behavioural trial to treat obesity (i.e. not based on a drug treatment), a team led by Anna L Ford at The Bristol Care of Childhood Obesity Clinic found that by retraining the eating habits of obese patients, sustained weight loss can be achieved.

The trial centres around a new technology called the Mandometer, which has previously been marketed as a device to cure Anorexia-Bulimia.

Mildly humorous instructional video.

The device is essentially a set of weighing scales, linked to a computer, which monitors how much you are eating and how fast you are eating it. Participants record how full they feel on a 100 point scale at various times throughout the meal, and the device then tells them to eat more slowly or quickly depending on their answers.

In recent years, we have come to redefine Anorexia-Bulimia as a behavioural or psychological rather than a medical condition. This study puts forward the argument that it may well be time to look at obesity in the same manner.

As the authors say

An intervention aimed at slowing down speed of eating and reducing portion size through retraining eating behaviour is a useful adjunctive therapy to standard lifestyle modification in obese adolescents.

Now it isn’t particularly surprising that when told what to eat, and how quickly to eat it, the trial participants lost weight. Nor is the conclusion that behavioural based eating interventions are a good way to sustain weight loss a real revelation. However this trial does offer some hope for making sure that patients retain a healthy weight after observation has ended, mainly because the device can be used at home and without supervision. Perhaps technology based solutions may provide a fruitful area of study in the future?

Posted in f1000, Literature, Medicine | Tagged: , , , | 4 Comments »

On the run-12Feb10

Posted by rpg on 12 February, 2010

Cancer Causes Cancer!

Well, that was the headline we should have gone with. It is of course a hat tip to the Daily Mail, a tabloid publication that is desperate to tell the UK population that just about everything causes cancer. (I found that website by googling ‘cancer causes daily mail’, which is in itself quite a neat headline. Unfortunately I think we’re closer to curing cancer than curing the Daily Mail. Oh well.)

So, we know that tumours have this nasty habit of sending out malignant cells into the rest of the body. They break off from the primary site and get into the blood and lymphatic systems, occasionally washing up in convenient organs where they can settle down and create new tumours, or metastases. This is partly why cancer is so difficult to cure: you can cut out the original malignant growth, zap it with X-rays and take all sorts of evil drugs (‘evil’ because they are designed to kill cells, and you’re made up of cells; and discrimination between the cancer cells and normal cells is a huge problem); but if one metastatic cell survives, you have to start all over again. And if it’s managed to find a home deep in a bone, or the brain, or somewhere equally inaccessible, it’s game over.

It turns out things are even worse than that. Circulating tumour cells, if they find their way back to their original ‘home’, can actually stimulate growth of the original cancer. Nasty. As the authors say,

Tumor self-seeding could explain the relationships between anaplasia, tumor size, vascularity and prognosis, and local recurrence seeded by disseminated cells following ostensibly complete tumor excision.

‘Ostensibly complete tumor excision’—that’s right, because no matter how good your surgeon is, you can never be sure you’ve cut every last bit out; or that some cells haven’t already gone walkabout.

The good news is that certain cytokines derived from the tumour, IL-6 and IL-8, act to attract the circulating cells, and that they get back in via the matrix metalloproteinase collagenase I (MMP-1) and fascin-1 (it’s the actin cytoskeleton again! These guys get everywhere). If we can find a way to selectively block these pathways we should be able to start thinking about appropriate therapeutic approaches. Gentlemen (and ladies), start your (grant-writing) engines.

Kim, M., Oskarsson, T., Acharyya, S., Nguyen, D., Zhang, X., Norton, L., & Massagué, J. (2009). Tumor Self-Seeding by Circulating Cancer Cells Cell, 139 (7), 1315-1326 DOI: 10.1016/j.cell.2009.11.025

Twitter storm

It’s been pretty hectic on the twittertubes this week. Following a random conversation at the Scholarly Kitchen I suggested writing papers in 140 characters would be a wheeze. I turned it into a competition, and we had an amazing response. Check back on Monday to find out who’s the lucky winner of a bag of f1000 swag.

Badger Wars

vermin shooting verminI don’t have a lot to say about badger culling to prevent/reduce bovine TB (except maybe to say that killing vermin with a high-powered rifle and decent ‘scope is one of the most humane ways of doing this).

I just like the sound of a ‘randomized badger culling trial’. Oh, and when someone ‘explains’

This trial was undertaken in very specific circumstances and it could be misleading to extrapolate the findings to any future control program.

you can be pretty sure there’s a vested interest or extreme prejudice somewhere. Even when the trial shows that there’s no economic benefit.
Jenkins, H., Woodroffe, R., & Donnelly, C. (2010). The Duration of the Effects of Repeated Widespread Badger Culling on Cattle Tuberculosis Following the Cessation of Culling PLoS ONE, 5 (2) DOI: 10.1371/journal.pone.0009090

Valentine’s Day

Just a reminder to all you chaps out there—it can’t hurt to buy some flowers, even if you don’t want to buy into the whole commercialization thing. A nice dinner doesn’t cost you much either, and could pay dividends in the romance stakes. But at the very least, show you really care by getting checked out:

Take a test for #Valentine‘s Day. Sexual health appointments across Lincolnshire within 48 hours. Call 01522 539 145

It gets pretty lonely up there in Lincolnshire. Have a good weekend, and I hope it’s full of lovehearts and kisses. Failing that, a beer or three can have much the same effect.

Posted in Friday afternoon, Literature, Medicine, Random, Science | Tagged: | 1 Comment »

On a new publishing model

Posted by rpg on 9 February, 2010

UPDATE: Entries so far

Twitter, what is it good for? Hunh.

There’s been rather an interesting couple of posts over at the Scholarly Kitchen, recently. What am I saying? They’re all interesting. Anyway, Kent Anderson says that blogs are for fogies and David Crotty talks about ‘talking’ vs ‘doing’. Elsewhere on Nature Network we’re re-visiting the meme of why do we blog anyway? (to which I’m not going to contribute, myself having decided to do rather than talk about). You can look up the links yourself if you can be bothered.

Anyway, in the middle of a rather long and involved conversation, someone made a throwaway comment on David Crotty’s post. Then I thought it might be fun to see if I could write a scientific paper in 140 characters.

“Clned gene _cancer_. KO in Ms. Ms dead. Cure cancer.”

But why stop there? Here’s a challenge for you.

Your task is to re-write a scientific paper, a real, peer-reviewed and published one, in 140 characters. Twitter it with the hashtag #sci140 so we can track them (OK, so that’s 7 characters you’ve just lost but no one said it would be easy). You can do this as many times as you like, as many papers as you like, and it would be nice if they were your own, but they don’t have to be. I’ll see if I can get some f1000 swag for what I deem to be the best entry.

Go for it.
Read the rest of this entry »

Posted in Friday afternoon, Literature | Tagged: , , , , , , | 2 Comments »

I’m a believer

Posted by rpg on 3 February, 2010

I took my daughters round the new Darwin Centre at the Natural History Museum last year. Because we happen to be friends, I managed to persuade the incomparable Karen James, of The Beagle Project fame, to show us behind the scenes.

Calamari at the NHM After that I took the girls into the ‘Cocoon‘, a huge butterfly egg-type structure that contains 20 million or more specimens, with superbly-done exhibits and displays. The Cocoon lets visitors see into the workings of the Museum; quite literally, because one side of the egg cuts away into the research labs (where Karen works). As we walked around we came across a video display of Karen herself, talking about the process of publishing science, how you write and revise a manuscript and send it off to the dreaded ‘peer review’.

Karen did a splendid job explaining the process: how other scientists in your field look at your work and—in an ideal world—check that you’ve done the work right and that you’ve cited all the relevant literature; that the manuscript is sound. She also conveyed, far too convincingly, the heartbreak of having a treasured manuscript rejected!

The process of science for the last three hundred and more years is based on peer review. Other scientists check your work and say yup, that looks OK or no, you need to do this other experiment or read these papers. Unfortunately some reviewers (and I stress, these people are peers, that is they are your equals; not some shadowy cabal curating or judging Science from on high) do seem to hold personal grudges, or have strange agendas, or simply not be very good. (And yes, sometimes your own work is pants and should be rejected. Deal with it.)

This leads to people making wide-ranging and inflammatory statements such as “peer review is broken.” Some of them even write letters about it (as reported by the Beeb, six months later). This leads to calls for making the peer review process ‘open‘; i.e. publishing the correspondence between the reviewers and the editors, and maybe even removing the anonymity part.

We’ve been here before (haven’t we always?):

All editors have seen curt, abusive, destructive reviews and assumed that the reviewer would not have written in that way if he or she were identifiable. Openness also links accountability with credit. One important defect of closed review is that reviewers don’t receive academic credit. Finally, openness should eliminate some of the worst abuses of peer review, where reviewersunder the cloak of anonymitysteal ideas or procrastinate

BMJ, 1999

but it’s not clear to me, and indeed the results of those BMJ studies tend to bear me out, whether this is really a problem, or whether the perception is far worse than the reality. And I’ve had a paper that took nearly two years to get published.

I remain to be convinced that peer review is broken. The idea of some sort of clique quashing acceptance of manuscripts isn’t that far-fetched, pace Philip Campbell. Strong editors will get round it, but in some fields it’s quite possible for one or two individuals to make it really difficult to get anything published. This tends to be self-limiting though: in the extreme case, the field simply dies. In my experience of this, the community know who those people were, so open peer review probably wouldn’t improve matters. Double-blind will not work because it’s going to be reasonably trivial to figure out is the author. People will still sit on manuscripts, and we already know who does this when it happens.

Making the reviewers’ comments, signed or not, public might not ‘improve’ peer review: however I do believe it has value. There is virtually no training in reviewing papers, and if young post-docs and grad students could see a wide range of reviews of many different papers, surely that can only improve their reviewing skills? Maybe it would even serve to make the first submission better if nascent authors were to look at reviews in their field, and discover the common mistakes?

Having said that, I am keen to see greater accountability. On a personal level, I wouldn’t write anything I wouldn’t put my name next to, and I don’t actually see why peer review should be any different. At f1000, for example, we already put our Faculty Members’ names on their evaluations, and we call this ‘post-publication peer review’. (Our motives are slightly different of course: we’re saying that you should take notice because of who’s writing them.)
The EMBO Journal has been experimenting with publishing reviews of accepted manuscripts for a year now. I was amused to find that when I clicked on one of the reviews at random, it was a paper about my old friend talin. And it’s from Mark Ginsberg and Iain Campbell, FRS, with both of whom have I coauthored papers.

Small world.

Posted in Literature | Tagged: , , | 6 Comments »

Come Home

Posted by rpg on 25 January, 2010

Back when I was an acolyte in the service of science, I worked on an interesting little big protein by the name of talin. This 270 kDa sucker is involved in focal adhesions: the ‘ankle’ of the cell, joining the actin cytoskeleton to the outside world. Focal adhesions are fascinating and complex, and if I had access to my thesis right now I’d draw you a picture. You’ll just have to take my word for it.

Talin is a molecule that back then was probably too big a problem for a grad student to tackle more or less single-handedly; the post-doc was was concentrating on some genetic analysis and my supervisor was taken up with the department’s computing services, which left very little lab time. I did manage to show that talin didn’t, as had been proposed, cap or nucleate actin polymerization (a negative result that was essentially unpublishable), and I also developed a long-term love of immunofluorescence microscopy. It appears we were ahead of our time: talin is closely involved with integrins (which see) and seems to be enjoying a (re-)surgence of interest lately.

Back to focal adhesions. Formation of these structures, completely essential to cell adhesion and migration, has been pretty thoroughly prodded. What’s interesting is how focal adhesions disassemble, so that the moving cell doesn’t get stuck. Again, this is something I had a professional interest in: one of my projects in Cambridge involved  the determination of how moving cells generate the required motile force, or how they put their ‘feet’ forward’. We used a model system and discovered that essentially it’s gel effects. You take a load of little rods (= actin filaments), grow them, and the space they fill is disproportionately large, driving protrusion. What we didn’t do was look at the trailing edge of the cell, how the ‘foot’ comes up again. That was something I would have dearly loved to work on, and if I’d stayed in Cambridge, or even in science, I might have worked on it.


Good job I didn’t, because we’ve just published an evaluation of a paper showing that focal adhesion disassembly is just as complex as assembly. It turns out that our old friend clathrin, along with two of its adaptors, gets directed to focal adhesions by microtubules, and, as you might expect seeing as clathrin is involved, the integrins are reincorporated into the cell by endocytosis, and recycled (rather than being left behind as the cell walks away. I don’t think any of us thought much of that hypothesis anyway, but I mention it for completeness.)

The researchers used total internal reflection fluorescence and watched individual molecules on the underside of migrating cells scooting around. The clathrin sidled up to focal adhesions, hooked up with integrin; and the two left the party together.

time series of leaving the partyGet your coat, you’ve pulled

Just as we don’t think that our spaghetti/copper wire/gel effects produce all the force required for forward motion, neither is it certain that all focal adhesion disassembly is driven this way (the paper says that depleting clathrin reduces disassembly by 60-80%) . Talin itself has a head domain and an extended domain, and there is a calpain protease recognition site at the join (this was an immense pain when purifying the native protein; I had to make sure everything was swimming in protease inhibitors). Similarly, it’s possible that calpain actively degrades one or more focal adhesion components to make sure the whole thing gets packed away nicely, even if that does seem expensive in energy terms.

It doesn’t stop there, of course. Somehow these integrins have to get recycled to the front of the cell. It would makes sense for the little blighters to make their way through known endocytotic pathways and be ready for reassembly into focal adhesions at the business end of the cell, this hasn’t yet been demonstrated directly. It’s probably a mass effect. He said, airily.

Read the rest of this entry »

Posted in Literature, Science | Tagged: , , , , , , , , , | 2 Comments »

On the run-22Jan10

Posted by rpg on 22 January, 2010

Last week I met up with a Certain Editor from a Certain Journal. We had a nice chat about, among other things, the policy of Certain Journals as regards the wind direction in the publishing industry. From the research side of the fence it’s easy to assume that publishing houses are monolithic edifices intent on maintaining a monopoly; unchanging and unfeeling; not to mention dirty money-grabbing bastards. Certain Advocates (not all!) of Open Access/Open Science hold to this view.

Naturally, the truth is a bit more subtle. Publishing is a business, which means that publishers are actually going to drive change, because if they don’t they will wither and die. Cell’s Article of the Future is a case in point (you might not like it, but you can’t deny that Elsevier are innovating). Publishers that see their competitors doing stuff are going to adapt and respond as necessary. Because if there’s a new paradigm brewing and they’re not on board, things will turn out bad for them. (The trick, of course, is to figure out what is actually a new paradigm and what is simply a tale/ Told by an idiot, full of sound and fury.)

Not unrelated to this preamble, you may have noticed a slight addition to the type of things we evaluate. For example, we have something titled Endogenous Forces Exerted at E-cadherin Based Cell-Cell Contacts:

The authors first determine that the traction forces that a single cell exerts on the underlying substrate are balanced across the cell. However, for cell pairs, it was determined that the traction forces exerted on the substrate for an individual cell did not balance, with the force imbalance reflecting the force exerted on the cell-cell contact.

but it’s not a paper being evaluated, it’s a poster.

Poster preview

I have more—much more—to say about this initiative, so stay tuned for updates.

While we’re on f1000, an evaluation you might have missed. (I’ve been meaning to blog about it all week, but as you’ll see it’s not really my fault. Or perhaps it is?) The piece, by Lutz Jäncke at the University of Zurich, starts provocatively:

One of the most virulent disputes between neuroscience and philosophy is whether human beings are equipped with a free will.

The brain is thought to generate decisions before the conscious mind gets involved, based on electroencephalography measurements. The ‘Bereitschaftspotential’ starts up to 1.5 seconds before movement is executed. However, in an elegant but mind-bending experiment published in Neuropsychologia, a diverse group of researchers find that there is a final check that seems to be under the control of the individual, an ultimate ‘yes or no’ that appears to be freely willed:

Planned actions can be subjected to a final predictive check which either commits actions for execution or suspends and withholds them. The neural mechanism of intentional inhibition may play an important role in self-control.

There goes my excuse that the voices in my head made me do it. You can read the evaluation, free for three months, here.

In other news, the dev team have been working on the search mechanism for the new f1000.com website. One of the major criticisms of the current Biology and Medicine sites is that it’s actually quite difficult to find the search bar, probably because it’s never in the same place. The new site is based around search (very Web 3.0, yeah?) and so it’s something that’s essential to get right. I think you’ll like what we’ve done, and once we get the design right I’ll post some screen shots. What I can tell you now is that you’ll have no problems finding it!

Finally, don’t forget to check out our Facebook page. Steve has been busy updating it with press release and video links, and we’ve noticed that several of you are reading the blog through it. Please feel free to ping us either here or on Facebook if there’s anything you’d like to see.

Have a great weekend!

Read the rest of this entry »

Posted in f1000, Friday afternoon, Literature, Website | Tagged: , , , , , , , , , , | Comments Off on On the run-22Jan10


Posted by rpg on 21 January, 2010

The distribution and uptake of antivirals and vaccination was in the news quite a bit before Christmas. H1N1 swine flu didn’t turn out to be the Armageddon some commentators were forecasting, but I don’t think it’s overstating the case to say that we dodged a bullet there. In cases like this we might expect the government to give a clear message, based on the best possible epidemiology. No, please, stop laughing. After all, the UKian government was right about the MMR combined vaccine, even if they did handle the situation incredibly poorly.

And that’s a problem, isn’t it? In our culture we don’t trust what the government tell us anymore. That may or may not be a good thing, but it certainly creates problems for public health policy, especially in a potential epidemic situation. Sometimes it’s quite clear what the right thing to do is, but how do we get them to do it?

A paper just published in PNAS and reviewed on f1000 (link free for three months) sets out an economic framework for controlling transmissible and evolving diseases. Now, I’m not an epidemiologist (you should possibly go and talk to my mate Bill if you’re that interested), and the argument therein apply more to a healthcare system that is not free at point of care (Obama’s reforms notwithstanding), but it’s an interesting paper nonetheless.

Antibiotic treatment for otitis media

“public policies such as taxing and subsidizing goods are frequently used to correct (for public benefit) the private actions of individuals when externalities, or side effects, of these actions exist.”

By comparing four different scenarios and addressing negative externalities, the authors predict where financial dis/incentives should be applied for maximum public health benefit. The scenarios discussed here are

  1. Tetanus: infectious but not transmissible between humans, and no herd immunity
  2. Measles: infectious, effective vaccination that generates a herd effective by reducing transmission
  3. Otitis media: non-transmissible, but unnecessary antibiotic treatment can lead to the negative externality of antibiotic resistance
  4. Pandemic influenza: antiviral treatment generates negative (resistance) and positive (reduced transmission) externalities.

Antiviral treatment for pandemic flu

There’s a whole heap of math in this paper, and although (or perhaps because) I’m supposed to be coming up with robust formulae for our rankings on the main site, it makes my brain hurt.

I find the thesis that economic impact can be leveraged to get maximum public health benefits an interesting one. I’m not sure how that would apply to the UK, for example, where the cost of healthcare is more-or-less invisible.

Infectious diseases tend to evolve quite rapidly when we attempt to control them, whether we use antibiotics, other drugs or prophylactic vaccinations. The framework expounded in this paper should prove to be readily applicable to a wide range of such diseases. Assuming, of course, that the authorities responsible for public health have the appropriate fiscal executive power, and access to current and accurate scientific information…

Read the rest of this entry »

Posted in Literature, Medicine | Tagged: , , , , , , , , , , , | Comments Off on Money

Private investigations

Posted by rpg on 20 January, 2010

One of the really great things about science is its potential for self-correction. If you have an hypothesis, a result (strange or otherwise), a set of data, it can be tested by anyone. This is encouraged, in fact: when you publish you’re not just saying ‘look how clever I am’ but also ‘here’s something new! Can you do it too?’. This philosophy is diametrically opposed to that behind Creationism, say; or homeopathy. In those belief systems whatever the High Priest says is of necessity true, and experiment must bend around them until the results fit.

This means that, in science, a finding or publication that people get very excited about at the time can be shown to be wrong—either through deliberate fraud, experimental sloppiness (although the boundary between the two can be fuzzy) or simply because we’re as scientists wiser now than we were then. This happens, and it’s normal and part of the process. We should welcome it; indeed, my friend Henry Gee has claimed that everything Nature publishes is wrong, or at least provisional.

So what we have to do is be completely open about this, no matter how embarrassing it is for the journal that published the work in the first place.

You know where I’m going with this, don’t you?

It was Derek Lowe who first alerted me to a paper published in Science last year, with the ome-heavy title Reactome Array: Forging a Link Between Metabolome and Genome. This was flagged as a ‘Must Read‘ (free link) back in November, because according to our reviewer Ben Davis

If this worked it could be marvellous, superb.

However, as Ben said in his evaluation,

this work should be read with some caveats. Try as we might, my group, as well as many colleagues, and I have tried to determine the chemistry described […] In my opinion, this is a work that deserves a “Must Read” rating and I strongly encourage the reader to read the source material and reach their own conclusions.

And as Derek points out, Science published an ‘Editorial expression of concern‘, noting a request for  evaluation of the original data and records by officials at the authors’ institutions, as well as mentioning it on their blog. Heavy. Immediately I saw this, I let our Editorial team know we might have a problem and we published a note to warn our readers that the work described in the paper was suspect.

Today we published a dissent to the evaluation from Michael Gelb, who says

There are many reactions shown that seem unusual and controversial […] My colleagues and I have tried to decipher the chemistry shown in Figure 1 of the main text and in the supplemental material. Many of the indicated reactions seem highly unlikely to occur, and the NMR data showing that some of the structures that were made are confusing and controversial.

We’ve also published a follow-up from Ben:

I agree wholeheartedly with the sentiments expressed in the Dissenting Opinion. The chemistry presented in this paper and in the online SI has varied in its description and content worryingly over the last 2 months.

and, rather tellingly,

as yet no chemical samples or key reagents have yet been made generally available.

(One of the usual conditions of publishing in reputable journals is that you make reagents available to other scientists, so that they can repeat your work. Failing to honour this commitment is not playing to the rules.)

It’ll be interesting to see when, not if, the original paper is retracted; and by whom.

And this, people, is the self-correcting wonder of science. Remember this, next time someone starts rabbiting about scientific conspiracies, or sends you their new theory of general relativity, or anything else that sounds crazy. It probably is.

Read the rest of this entry »

Posted in Journals, Literature, Science | Tagged: , , , , , , , | 3 Comments »