Friday 24 October 2008

Bad Science by Ben Goldacre

A bit off-topic this week in that the book I want to talk about isn't, by most definitions, science fiction or fantasy. However, my excuse is that it includes elements of science, fiction and fantasy – even though it is a non-fiction book about health care!

The author of Bad Science is a doctor working in the National Health Service who also writes a column for a national newspaper (The Guardian) and maintains a website on the same subject. His topic is the way in which the public is misled by claims made about medications and medical treatments; not just by practitioners of alternative and complementary medicine (including nutritionists) but also by pharmaceutical companies and, most of all, the news media. His writing style is journalistic rather than academic and is often hilariously trenchant as he names and shames some very well-known individuals, companies and newspapers. He must have some hard-working lawyers!

Bad Science is not just an entertaining read, it is highly informative. Goldacre not only provides lots of evidence to back up his denunciations, he constantly drives home the essential message concerning medical trials: that many of them do not achieve the quality "gold standard", for which there must be a placebo-taking control group; there must be a completely randomised selection of those in the trial and control groups; and the trials must be conducted under double-blind conditions (that is, neither the participants, nor those doling out the pills to them, know who is getting the trial medicine and who is getting the placebo). Failure to follow these precepts has been shown to have a major distorting effect on the outcome, and the author gives many examples of this.

In the first part of the book, Goldacre's critical eye is turned onto alternative medicine in general and homeopathy in particular. He shows that there is a direct relationship between the results of the trials in this field and the rigorousness of the way in which the trials were conducted. Put simply, trials which are conducted in accordance with the "gold standard" precepts listed above show that homeopathic pills work no better than placebos. As he points out, the link between the mind and the body is powerful and complex, and still not entirely understood (sometimes patients report feeling better even when they have been told that they are being given sugar pills instead of medicine!). Any benefits from alternative medicine appear to be in the ritual associated with them, as with shamanistic magic, which convinces customers that this must be doing them some good.

Next in his firing line come the nutritionists and peddlers of vitamin pills. The author's message is this: to give yourself the best chance of enjoying good health, eat a variety of fresh fruit and vegetables; drink alcohol only in moderation; don't smoke; and take regular exercise. He says that's basically all there is to it; there is no valid evidence that "superfoods" work any better than other fruits and vegetables, or that you can further improve your health by taking vitamin pills or dietary supplements. Amazingly, despite the almost universal acceptance of the idea that taking fish-oil pills for omega-3 fatty acids is good for brain development, there is no trial evidence which supports this for the general population.

Goldacre also takes a swipe at the illogicality of the "detoxing" and "antioxidant" fads, for which he argues that there is no clear case (in fact, the most exhaustive trials of antioxidant vitamin pills show a slightly increased chance of death compared with control groups taking placebos). He points out the tendency of all purveyors of quackery to "cherry-pick" the results of trials, quoting only the few (usually less rigorous) ones which appear to support their claims while ignoring the vast quantity of much larger and more reliable trials which demonstrate no effect beyond placebo.

In the light of the above, one of the intriguing aspects of this subject is the very high profile in the news media (including quality newspapers and the BBC) of alternative medicines, superfoods and vitamin pills, when there is no valid evidence that they work, and plenty that they don't. Goldacre explores this issue too, pointing out the money spent on promotion, the relentless efforts of the practitioners in getting their message across, and their immediate and often aggressive responses to any criticism. What he finds more shameful is that some universities have fallen into the trap and are offering courses in these fields, possibly blinded to the lack of any valid scientific basis by the popularity of the subjects and the universities' need to put on courses which will attract students and earn money.

So why are these "remedies" so popular? Partly, it seems, because we like the idea of a pill which will make our problems go away. The author quotes a recent large-scale trial of whether better parenting techniques could improve the behaviour of problem children, which resulted in dramatic benefits. Yet this was ignored by the news media, which constantly focus on pills and diet instead. I suspect that this may be because parents don't like to consider that it may be their failings which have caused the problems, nor that correcting them will involve a considerable effort over a long period of time. How much easier it is to be able to blame something else, and to dish out a magic pill for it.

The major pharmaceutical companies ("big pharma") are next up for attack. First, the introduction of new medicines to solve real diseases is slowing considerably, so the companies are inventing new medical conditions which their existing medicines are claimed to treat. He quotes "social anxiety disorder", "female sexual dysfunction" and "night eating syndrome" as three examples of problems for which big pharma are peddling their wares, even though they are probably not best treated by taking a pill. Much of the effort in developing new drugs is on devising variations of existing drugs which are sufficiently different to establish a new ten-year period during which the companies have sole rights to make them. And of course, big pharma has no interest in developing cures for the killer diseases which only affect the Third World (no money in them) nor in promoting effective and cheap remedies which use common ingredients which can't be patented (no money in those, either). To be fair, I don't blame the companies for these last two issues – as Goldacre points out, they are profit-making businesses, not charities – but they do highlight the need for some method of funding the companies (or somebody) to carry out such unprofitable but important work.

The author describes in detail the way in which big pharma can distort the results of trials of their new products. For a start, such trials are funded by the companies themselves, and he quotes strong statistical evidence that the source of funding biases the results in favour of the funder. He then describes in considerable detail the different techniques which big pharma uses in order to present trial results in the best possible light (including burying negative outcomes) to get approval for their products and to persuade doctors to prescribe them. Even without such deliberate manipulation, negative outcomes of trials are less well-reported than positive ones anyway, an effect known as "publication bias". All of this results in medicines being presented as far more beneficial than they really are (in fact, it was recently reported in New Scientist that some well-known medicines don't work at all, or work far less well, when patients are not told what they are being given and what they are supposed to do).

Despite all of the above, Goldacre's major criticisms are focused on the news media for the way in which they report science in general and medical science in particular. He takes as examples two recent high-profile issues in the UK; the prevalence of MRSA in hospitals, and the alleged association between MMR vaccinations and autism. The first story was fuelled by positive MRSA readings from hospital swabs taken by undercover journalists, but it transpired that these were processed by a man unqualified to do such work, in an amateur lab in his garden shed. Those samples which were double-checked by proper labs produced different results. Yet the newspapers continued to use the amateur lab because they knew that this would provide positive results, which made for a better scare story (which is not to say that there isn't a problem with MRSA in British hospitals – but its scale was exaggerated).

The attitude of many newspapers to the MMR scare was even more shameful; they accepted the word of one doctor that there was a link with autism, on the basis of one very small-scale study which didn't even meet the minimum standards for a trial. They gave this at least equal weight to the assurances given by the medical profession that there was no link, and ignored huge and properly conducted large-scale studies which demonstrated this. In fact, one newspaper waged a vicious campaign against MMR for several years, until many people were left feeling that there must be something in the story, and that the government and the medical establishment were just trying to cover it up. As a result, take-up of the MMR jab has declined to the point that measles, mumps and rubella, which had been on the way out, are making a comeback.

Perhaps the most depressing aspect of the news media reporting is not that some lower-quality papers uncritically report bad science as if it were fact, but that even the best newspapers – and the BBC, heaven help us – are all guilty of bad science reporting. Most of them have science correspondents who know better, but once a major and exciting (i.e. controversial) science story breaks, the job of covering it is often given to higher-status mainstream reporters who are ignorant of science and proper trial protocols, dumb down the issues, and sometimes ignore the conclusions of scientific studies in order to dig into the data to find nuggets which they can take out of context to fuel the conspiracy fires. While it can be argued that the news media are not entirely to blame for all of this (after all, they only do it because lots of people want to read about such things – we get the news media that we deserve) it is highly irresponsible for them to whip up unfounded panic, resulting in serious medical consequences, in the interest of sales.

One result of all of this "bad science" is an appalling lack of understanding of science, including basic statistics, throughout society. The author quotes a couple of criminal trials which resulted in the defendants being imprisoned for murder on the basis of highly questionable statistical evidence (one of them was later released).

The only criticism I have of the book is that the characteristic which makes it such an entertaining read – the author's unbridled attacks on what he considers to be bad science – also mean that it is more of a polemical work than a balanced account. However, the evidence and analysis which Goldacre provides is compelling, and this is a book which everyone should read. It is similar in its theme (although most of the content is different) to Gilovich's equally important book How We Know What Isn't So, reviewed in this blog on 1 August this year.

1 comment: