Why Honey Isn't A Cure

By David Colquhoun

Uhuh, here we go again. All over the media we see headlines like “Honey ‘beats cough medicine’.“ Take for example, the Daily Telegraph, where Ben Farmer writes “Honey is better at treating children’s coughs than an ingredient used in many over-the-counter medicines, according to new research”. (Read the press release)

That is NOT what the research found. This is what the research paper itself says (DM refers to the standard ‘cough suppressant’ dextromethorphan, which is already known to be ineffective).

“honey was significantly superior to no treatment for cough frequency’

DM was not better than no treatment for any outcome.

Comparison of honey with DM revealed no significant differences.”

See it? No detectable difference between honey and standard cough medicine.

At first sight, the results seem contradictory; no difference between honey and DM; no difference between DM and ‘no treatment’. So how can honey be better than ‘no treatment’?

What was done?
Three things were compared: (a) buckwheat honey, (b) a standard ‘cough suppressant’, dextromethorphan in a honey-flavoured syrup that was designed to be similar to the honey (DM for short), and (c) no treatment whatsoever.

Thirty-five patients received honey, 33 received DM, and 37 received no treatment.

The good thing is that the treatments were allocated randomly to the children, and that the person doing the assessment didn’t know which treatment each child had received. The children didn’t know whether they were getting honey or DM either, but they DID know when they got ‘no treatment’.

The trial was carried out over two days. On day one nobody got a treatment, but they filled in a survey that asked, for example, “how frequent was your child’s coughing last night”. The parent had to tick one of seven boxes, from ‘not at all’ (score zero) to ‘extremely’ (score 6). They were then given the treatment allocated to them in a brown paper bag, so the person who gave it didn’t know which it was. The patients then went home and on the next day the same survey was completed by the same parent, over the telephone.

What happened?

First look at the raw data. Here is Figure 2 from the original paper (click thumbnail to enlarge).

The first thing to notice is that there are no error bars on these charts. In my area, at least, no journal would accept a chart like this with no indication of scatter.

It would have been better to have a chart in which the difference in score between day 1 and day 2 was calculated from each patient, and the size of these differences plotted, with a standard deviation of the mean to indicate the amount of scatter in the observations. I have asked Dr Paul to send me a version that indicates the scatter of the numbers in this way (but I don’t think it will come).

The second thing to notice is that there is there is quite a big difference between the score on the first day (pale columns) and on the second day (dark columns), even in the no treatment group .

Thirdly, the pale columns are all much the same. On the first day the average score was about 4 (”a lot”) though on the second day, even with no treatment, the score fell quite a lot, to something between 2 (”a little”) and 3 (”somewhat”). This is a bit baffling because no treatment was given on either day. Presumably it results from the different settings in which the survey was given, or because the kids were getting better anyway.

Fourthly, insofar as the pale columns (baseline values) are all much the same, the thing you need to concentrate on is the difference, on each chart, between the height of the dark bars, for honey, DM and no treatment. These differences are pretty small, but on all the charts, the honey score is slightly smaller than the DM score, and the DM score is slightly smaller than the ‘no treatment’ score. What are we to make of that?

About the Author

David Colquhoun FRS is professor of pharmacology at UCL who regularly debunks the claims of homeopaths and 'alternative therapists' in his excellent blog Improbable Science.

Here beginneth the statistics lesson
Put roughly, we have to ask “how probable is it that the observations could arise by chance”. More precisely, the question is this. If there were no difference between the treatments, what is the probability that we would observe by chance a difference as big as, or bigger than, that seen in the experiment? (You need the subjunctive mood to explain statistics - pity it’s vanishing.)

Above each chart in the figure we see P < 0.001. This means that there is less than a one in 1000 chance of the results arising by chance. More precisely, if all three treatments (honey, DM and no treatment) were actually identical, it is very unlikely that we’d see these results. The reasonable conclusion is, therefore, that all three treatments are not identical. The problem with this argument is that it tells you nothing about where the differences lie, so it is of no help whatsoever to a patient who is trying to decide what to do about a cough. The other problem is that it includes the ‘no treatment’ group, which was not blind. Both the children and parents were well aware that no treatment was given.

The most helpful comparison is really the properly-blinded comparison between honey and DM. And when this was looked at the result was no significant differences. In other words the small differences between the heights of the dark columns for honey and DM could perfectly well have arisen by chance if honey and DM were identical in their properties.

There isn’t any reason at all to think that honey is better than the standard (but ineffective) cough medicine.

The ‘no treatment’ group certainly has some interest, but the fact that it was not blind means that the fact that honey was marginally better than ‘no treatment’ could perfectly well mean that taking honey has a better placebo effect that doing nothing at all. It provides no evidence at all that honey has any genuine therapeutic effect. If it had, one would then have to find out if the therapeutic effect was specific to buckwheat honey, or whether any old honey would do. It could be argued that even if the effect were real rather than placebo, the size of the effect is too small to make all that effort worthwhile.

A couple more things
It is already well known, from several good studies, that DM is useless, no better than placebo. This inconvenient fact has not yet reached many places that it should have (not even mentioned on wikipedia for example), but the American Academy of Pediatrics says

“Numerous prescription and nonprescription medications are currently available for suppression of cough, a common symptom in children. Because adverse effects and overdosage associated with the administration of cough and cold preparations in children have been reported, education of patients and parents about the lack of proven antitussive effects and the potential risks of these products is needed.”

The discussion in the paper by Paul and his colleagues, seems surprisingly upbeat about honey, in the light of their own findings. I’m surprised that they use the term ‘demulcent’ which I had thought to have died out, like the word ‘tonic’, on the grounds that it had no defined meaning

It is because meaningless terms and useless medicines die out eventually that medicine makes progress. The problem with alternative medicine is that nothing dies out: on the contrary they keep adding myths.

And finally
Always look at the end of the paper. On this one we see that the study was paid for by the National Honey Board. Dr Paul assures me that the funding source had no say in the design or analysis, which is as it should be.

Financial Disclosure: Dr Paul has been a consultant to the Consumer Healthcare Products Association and McNeil Consumer Healthcare.

Funding/Support: This work was supported by an unrestricted research grant from the National Honey Board, an industry-funded agency of the US Department of Agriculture.

So what is the practical outcome?
My conclusion from all this is simple. If you have got a cough, tough luck. There isn’t really anything available, conventional or alternative, that does much good. You’ll just have to wait for it to get better. But if you want to take something that tastes nice, why not honey? It almost certainly won’t do any good but it tastes good and it’s safer than the standard cough medicine.

Get more on this at Prof. Colquhoun's truly excellent blog Improbable Science whence this article has been adapted with kind permission from the author.