Faculty of 1000

Post-publication peer review

Here we go again

Posted by rpg on 19 January, 2010

CAM is a group of diverse medical and health care systems, practices, and products that are not generally considered to be part of conventional medicine. While scientific evidence exists regarding some CAM therapies, for most there are key questions that are yet to be answered through well-designed scientific studies—questions such as whether these therapies are safe and whether they work for the purposes for which they are used.

So says the US National Institutes of Health about ‘complementary and alternative’ medicine. What the NIH doesn’t tell you is that some people get quite ratty, and indeed irrational, when talking about it. It also fails to point out that that if something doesn’t work it can hardly be called ‘medicine’, so the phrase ‘complementary and alternative medicine’ is a bit pointless.

Yes, I know: CAM means anything that isn’t ‘Western’ medicine, the stuff that often has huge amounts of evidence behind it and that actually makes people better. Or maybe anything that isn’t pouring money into the coffers of multinational pharmaceutical companies—as if that’s somehow worse than pouring money into the coffers of those who offer sugar pills and countless different vitamin formulations.

Anyway, the point is that anything that claims to be a medicine can be tested, to see if it actually does make people better. Isn’t that what we’re trying to do? Anyone who tells you that a CAM therapy can’t be tested is lying to you, and by the way, when did you last see your wallet? The endpoint for all medicines or treatments is really quite simple: does someone who is unwell get better as a result? Where things get complicated is in the testing protocols you use: in defining ‘better’; in the appropriate choice of controls; in statistical analysis; in dosage; and indeed in defining what ‘unwell’ means in each case. (It’s trivial to feel a bit run down, take some supplement or do yoga and feel better again, and say that the intervention worked; when in fact you had just had a couple of bad nights’ sleep. This is why the plural of ‘anecdote’ isn’t ‘data’.)

More problems arise when people start getting lazy and lumping together things like herbalism and acupuncture with homeopathy. One of those involves chemicals extracted from plants, one involves a surgical procedure; and one nothing at all. It makes no sense at all to compare them as equals. (Now you might say that homeopaths treat the patient and not the condition, and you might well be right; but conventional medicine also has that. It’s called ‘bedside manner’ and it’s very powerful.) Now, I have no particular beef with homeopathy per se: if educated, well-informed people want to try it, then that’s their lookout. But when people in positions of authority or responsibility actively deny access to ‘conventional’ medicines that actually work, in favour of witchcraft, I get a bit tetchy.

We talked a week or two ago about a study that looked at a herbal remedy; chamomile. There, chamomile came out as beneficial in cases of mild anxiety. This was a nicely done trial, with a result that wasn’t really surprising. After all, people have been using it for ages (and noticed an effect; presumably if they hadn’t they’d stop buying it) and there’s actually some chemical in it that has a chance of being biologically active. Chamomile tea might well be classified as a CAM, and it works. We can call it ‘medicine’, therefore, even if it’s not prescribed or ‘conventional’.

A study reviewed on f1000 this week examined acupuncture in the management of post-operative pain in children. Again, actually doing something to people had an effect; and this is with anaesthetized patients (so you might think that patient bias is reduced). This reminds me, parenthetically, of my favourite placebo story. Back in the Fifties a particular treatment for angina involved ligating certain arteries. When people did the proper experiment, they found that a sham procedure (under sedation, not anaesthetic) was just as effective, at least in the short term. Rather than say “hey, we’ve discovered something odd here” the surgeons stopped doing the procedure (that’s because they were medics, I presume. Were they scientists they might have played with the observation a bit). This also is not too surprising: acupuncture seems to work. We might not know how it does, but at least we can guess that the mechanism involves stimulation of certain nerves or release of endorphins, or something testable (but probably not some mysterious ‘life energy’):

For short-term outcomes, acupuncture showed significant superiority over sham for back pain, knee pain, and headache. For longer-term outcomes (6 to12 months), acupuncture was significantly more effective for knee pain and tension-type headache but inconsistent for back pain (one positive and one inconclusive).

The accumulating evidence from recent reviews suggests that acupuncture is more than a placebo for commonly occurring chronic pain conditions. If this conclusion is correct, then we ask the question: is it now time to shift research priorities away from asking placebo-related questions and shift toward asking more practical questions about whether the overall benefit is clinically meaningful and cost-effective?

And then we have the homeopaths. Bless them: although they say we can’t, or shouldn’t, test homeopathic ‘treatments’ like we test other CAM, sometimes they’ll go ahead anyway. And they’ll say things like

Piglets of the homeopathic treated group had significantly less E. coli diarrhoea than piglets in the placebo group (P < .0001).

Which, if true, would be brilliant.

But it’s not, is it? This is why they’re publishing in Homeopathy and not somewhere like Nature (because believe me, if it were true this would be Big News). Even so, that’s a remarkable claim, and I had a look at the paper to see what was going on.

It’s really rather simple. They treated (observer-blind, apparently. I guess the pigs didn’t know what they were getting) sows with either an homeopathic preparation or a placebo (or maybe vice versa. Hard to say). And then they scored the offspring of these sows for E. coli-caused diarrhoea. The piglets that had diarrhoea that wasn’t caused by E. coli didn’t get counted. Funnily enough, those pigs were all in the treatment group, which is how the authors managed to get a sixfold decrease and that rather splendid-looking p value. And how did they determine that those pigs weren’t suffering from that particular form of the squits? Well, it wasn’t by microbiology:

Faeces were cultured to identify enteropathogenic E. coli, E. coli K99 and Salmonella. None of these were identified as present in the faeces sample. This does not per se demonstrate that enteropathogenic E. coli were not present at the farm at that moment. It was a relatively small sample size of three litters, which would not necessarily include the infective agent. Because treatment with Coli 30C had worked before, and E. coli diarrhoea generally can be distinguished based on day of appearance and colour, this was not further investigated.

Translation: “We believed there was E. coli in these pigs. We failed to prove it, but that doesn’t matter because we are strong in the Force, and you should believe it too. And there was no way we were going to check the whole batch, because then we might find an inconvenient absence of what we want to prove.”

And seeing as they didn’t publish the raw data, who are we to argue anyway?

It’s hardly any wonder, with work of this ‘quality’ finding its way into the literature, that homeopaths are still making a fast buck through the gullibility of ordinary people. So I heartily commend to you the 1023 ‘Overdose’ event on Saturday 30th January. This is where a bunch of community-minded people are going to, en masse, swallow a whole bottle of some homeopathic pills to proof they’re ineffective. And to reward you, loyal reader, for making all this way to the end, here’s a joke:

Did you hear about the homeopathy patient who died of an overdose?

He forgot to take his medicine.


2 Responses to “Here we go again”

  1. Jim said

    I am still just utterly befuddled that they could even believe they could get away with saying:

    “None of these were identified as present in the faeces sample. This does not per se demonstrate that enteropathogenic E. coli were not present at the farm at that moment.”

    …doesn’t mean they’re not present in the farm, pfft. I guarantee we could find Clostridium tetani on that farm somewhere, do we infer they may have tetanus too?

    If I were Elsevier, I would be deeply ashamed of supporting this hack of a journal.

  2. LED said

    I particularly like a certain Australian comedian’s take on CAM in his beat poem:

    “By definition”, I begin
    “Alternative Medicine”, I continue
    “Has either not been proved to work,
    Or been proved not to work.
    You know what they call “alternative medicine”
    That’s been proved to work?

    It’s called “Storm” – worth a look at!

Sorry, the comment form is closed at this time.

%d bloggers like this: