Dr David Bender, chairman of HealthWatch, told BBC health correspondent Jane Dreaper, “The AIDS epidemic in Africa has been massively exacerbated by a refusal to accept the scientific knowledge about the disease.” [1] The workshop, which marked World Aids Day, was reportedly attended by about 80 homeopaths and workers from HIV projects. Its publicity had claimed it would offer “a fascinating insight into the role of homeopathy in treating HIV/AIDS” [2] and discussed the work of homeopaths in AIDS clinics in Africa and included a presentation on remedies with which, the conference was told, “the AIDS epidemic can be called to a halt”. However Dr Michael Brady, consultant in genitourinary medicine and HIV at Kings College Hospital and medical director of The Terrence Higgins Trust, supports HealthWatch’s objections, saying, “There is no credible evidence that homeopathy is an effective treatment for HIV. Anti-retroviral therapy is the only treatment that has been clinically proven to improve immune function, delay disease progression and prevent death in patients with HIV. It is misleading, irresponsible and potentially dangerous to suggest otherwise.”3 Dr. Ian Williams, Consultant in Genitourinary Medicine at University College London agrees, “There is no proven evidence that homeopathy will treat HIV infection effectively, nor is there a biological plausibility this may be the case.” [3]
But the people of SubSaharan Africa are desperate and vulnerable: the continent
has 22.5million people infected with HIV, accounting for 68% of the global total.
As few as one in ten of them are being treated with retroviral drugs, and death
rates are high amongst those who cannot get them [4].
Consultant clinical scientist and HealthWatch committee member Les Rose heard
Society of Homeopath’s vice chair Jayne Thomas, interviewed on the BBC
Today programme on Saturday morning. “She said that the purpose of the
conference was to evaluate the evidence,” recalls Les. “But if that
was the case, I would like to know if any specialists in clinical evidence attended?
Will the proceedings be published in the open literature? Why use a conference
to evaluate evidence, instead of submitting papers to rigorous review in the
journals?”
Ben Goldacre, the Guardian’s “Bad Science” commentator and
last year’s winner of the HealthWatch Award, discussed the conference
in a column on “Aids quackery” where he quotes Society of Homeopaths
conference speaker Peter Chappell as saying, “Right now, Aids in Africa
could be significantly ameliorated by a simple tune played on the radio.”
[5]
1. http://news.bbc.co.uk/1/hi/uk/7122370.stm
2. http://www.homeopathy-soh.org/for-homeopaths/documents/Aidsflyer.pdf
3. Quoted in HealthWatch press release, 30 November 2007.
4. UN Aids. 2007 AIDS epidemic update. View on http://www.unaids.
org/en/HIV_data/2007EpiUpdate/default.asp
5. http://www.guardian.co.uk/commentisfree/2007/dec/01/aids
WoW is the strap line for Wellbeing of Women, a UK charity that works in partnership with the Royal College of Obstetricians and Gynaecologists (RCOG) to solve health problems that affect women by funding medical research and training into all aspects of reproductive health (see http://www.wellbeingofwomen.org.uk). They’re good: in the last 40 years they have invested over £27million to fund medical research and training. The charity was formerly known as Birthright and in the 80’s its patron was Diana, Princess of Wales.
The leaflet tells us that the show incorporates a “serious bit” which offers information on all areas of women’s reproductive and gynaecological health. It offers a chance to speak candidly to experts about health problems that affect you. The “fun side” is about, “Physical & Wellbeing Health...this is your chance to hear and sample a variety of complementary therapies,” and the list includes homeopathy, craniosacral therapy and Pilates. The show was held at the RCOG in September and is mentioned in the RCOG news which is distributed to all fellows and members of the college.
To my mind the RCOG is supporting, promoting, and possibly endorsing aspects of complementary medicine for which there is no sound evidence of efficacy. Does this matter? Why have I become so concerned that I have bothered to write this article and made myself unpopular at work by suggesting the posters should not be displayed?
On one level I am annoyed that my hard earned cash, some of which I have to
pay as a subscription, is being used to promote non-evidence-based techniques.
But more seriously I am troubled. I spent three years in full time clinical
research. I have experienced the triumphs of seeing my work published in peer
reviewed journals and have presented to national/international conferences.
I have also experienced the frustrations which anyone who has conducted research
will know all too well and the disappointment when papers are rejected. Many
colleagues and friends have dedicated years of their working lives to produce
ethical and sound scientific research, which will further our knowledge and
understanding. Yet this publicity appears to suggest that the RCOG gives as
much credence to complementary medicine, where there is little or no evidence
to support its claims, as it does to evidence-based conventional medicine. It
is, however, of interest to note the Wellbeing of Women’s and the RCOG’s
disclaimer at the end of the leaflet, stating that “…neither Wellbeing
of Women nor the RCOG can be held responsible for the validity of clinical treatments
or medical statements made.”
Hurt pride? Probably. Jealous I have been overlooked as an expert? Possibly.
But perhaps my most important concern is over the response I received at my
workplace. Receptionists reading the leaflets thought they should be given out
to patients. My attempts to explain my concern were met with hostile stares,
silence and were ultimately ignored. The posters remain on the walls and the
leaflets are distributed. I was considered puritanical and out of touch. After
all if it was being held at the RCOG it must be OK.
The attitude of my colleagues demonstrates the need for better education of healthcare professionals about critical appraisal of technical and publicity material relating to healthcare/new treatments; in this respect, the work of HealthWatch is more relevant than ever.
I remain disappointed in the RCOG.
Dr Gillian Robinson
Associate Specialist in Sexual and Reproductive Health
St Giles’ Hospital, London
HealthWatch’s 19th Annual General Meeting met on 18th October 2007 at the Medical Society of London, Chandos Street, London W1. The HealthWatch Award was presented to Professor Raymond Tallis for his thoughtful promotion of evidence-based medicine. He has been kind enough to supply us with the text of his presentation on the curse of the media anecdote. You can read this below.
The 2007 HealthWatch Student Prize for Evaluation of Clinical Research Protocols continues to generate interest in quality research amongst young medical and nursing students. This year’s Medical Student winner was Alison Myers, of the Royal Free and University College Medical School, who received a cheque for £500. Alison has agreed to offer her services to the HealthWatch Committee as a student representative now that her predecessor, Elizabeth Fairfax, has qualified as a doctor. Runner-up medical students were Alexandros Georgiou of Liverpool UMS; Ruth Heseltine of Barts and the London; Philip Jones of Nottingham; Gillian Norrie of Aberdeen and George Peck of Imperial College London. Top prize from the Nursing Students category, with another £500 cheque, went to Sarah Linge of Edinburgh. Lisa Common was a runner-up.
From time to time we would like to be able to contact members between Newsletters with alerts about new activities, important media reports, or to call for action. If you are willing to be contacted by e-mail, please send an e-mail to the membership secretary so that he can add your address to the list. Important: this list will only be used to circulate matters believed to be of genuine interest to HealthWatch members. The list will be confidential to the Membership Secretary and will not be used for marketing activities of any description.
Membership subscriptions to HealthWatch simply do not cover the cost of producing, printing and distributing the Newsletter four times a year, and therefore with regret the meeting agreed that subscription rates should be increased as follows:
I
will start with a quotation from an editorial in the Journal of the American
Medical Association of November 15th 1913 entitled Braughan’s pellagra
remedy: worthless nostrum sold under fraudulent claims. “Nostrum fakers
are damned. It is they who feed carrion-like on the fears of suffering humanity.
To those stricken with a well-nigh incurable disease they hold out the hope
of a sure cure.” This remedy contained quinine and iron salts, which give
a bitter taste, which would suggest to the pellagrin that it was good medicine.
Little seems to have changed over the last century, and HealthWatch seems to
be needed as much as ever.We publish this newsletter four times a year, and occasional position papers (with one revised paper and three new ones published this year), as well as maintaining a website with an archive of all past newsletters. In addition to this we have recently written to organisations such as the Medicines and Healthcare products Regulatory Agency and the Royal College of Obstetrics and Gynaecology concerning their promotion of unproven complementary and alternative therapies, with the aim of publishing our letters and their responses. We also hold the annual student prize for critical appraisal of clinical research protocols, generously supported by a grant form the Ajahma Trust, and make an annual award to an outstanding medical scientist, practitioner or journalist for his or her contribution to better understanding or practice of evidence-based medicine. We provide a source of information and informed comment for press, radio and TV.
The committee has spent much time over the last few months wondering what more we should do. Three areas have emerged:
Our great strength is our independence. Our only sources of income are membership subscriptions and occasional donations, as well as the grant from the Ajahma Trust for the student prize. This does not stop vilification that we are “in the pockets of big Pharma”, despite the fact that we receive no money from industry at all. We maintain friendly relations with Sense About Science; we share many of their aims, but not their more political objectives and we lack their financial and political resources.
We seem to have influence beyond what our small size might suggest. The website has received an average of 53,000 hits per month over the last year. About half of these are from addresses such as .com, .net or of unknown origin; of the half that can be traced to a country, only 16% are from uk addresses, and at least 50 hits in any one month have come from 50 different countries. One third of the hits come from people going directly to the HealthWatch website, and most of the rest come from Google, Yahoo and MSN searches, but a significant number come from Wikipedia –obviously we are considered to be an important, authoritative and independent source of information on health.
The Chairman is only as good as his committee, and I have to thank all members of the HealthWatch committee for their help. I must pay tribute to the late Michael Allen, who was a founder member of HealthWatch and over the years has done all the difficult jobs (treasurer, newsletter editor, secretary, mailing out the newsletter, and as his final gift to HealthWatch, reconciling the membership records held by the membership secretary and treasurer, which had diverged over the years). Anne Raikes now keeps the accounts in order; John Garrow is an indefatiguable vice-Chairman and contributor to the newsletter; Malcolm and Diana Brahams are always ready to offer legal advice, and Malcolm has written a number of solicitor’s letters on our behalf (we probably could not have afforded to pay a solicitor to do so); John Illman, Keith Isaacson, James May, Caroline Richmond and Les Rose are always ready to write for the newsletter; Gillian Robinson and Walli Bounds produced the protocols for the student competition and judged this year’s record entry.
There are four other people we could not do without: Mandy Paine produces this excellent newsletter every quarter; Caroline Addy reads the draft newsletter to check it for libel, pro bono; Joan Gandy undertakes all the work associated with running the student prize, and consistently refuses to accept even a modest honorarium; Kenneth Bodman took over as membership secretary (with the added burden of distributing the newsletter each quarter) in January, in return for a modest honorarium.
David A Bender
Now members can join the lively online discussions that are increasingly taking place between HealthWatch committee members and their colleagues. We have begun a forum site linked to the HealthWatch website. Anyone can view the discussions, but you have to go through the quick registration process in order to start or reply to posts in the forum.
It is expected that the HealthWatch Forum will grow to become a useful resource for members, a significant source of information, views and links related to HealthWatch and its aims, and an interesting, thought-provoking browse.
From the home page of the HealthWatch website read down the list of options on the left of the page, click the last one titled, “Member’s Forum”, then just follow the instructions and dive in.
Let
me say straight away how honoured I feel to here tonight to be talking to you
and to be receiving your annual HealthWatch Award. This came totally out of
the blue – which has very positively altered my attitude to the blue as
a result. The blue is more usually a source of nasty surprises – the frozen
block of urine dropped from a 747 – than nice ones. I feel not only honoured
but distinctly undeserving, particularly when I think of the previous winners
of your award. I am thinking of writers whom I admire enormously such as Geoff
Watts and Polly Toynbee and the witty and courageous John Diamond; and great
clinical scientists such as David Sackett and Richard Doll and Michael Baum.
And as for Ben Goldacre – the recipient of last year’s award –
what can I say, except that every time I read one of his columns, I punch the
air with delight until it is bruised. When get my weekly fix of Bad Science
all my exasperation evaporates. So thank you Ben, and thank you all my predecessors,
for strewing a little broken glass in the path of charlatans, mountebanks, loblolly
men, and all those alternative practitioners whose loud noises have threatened
at times to drown the softer-voice of hard-won scientific truth. And thank you
for inviting me to join their select company. I would like to add my congratulations
to the students for their awards. ‘O brave new world that hath such people
in it…’ Thank you also Nick and David and you other HealthWatch
stalwarts who over the last 19 years have kept HealthWatch as a major force
for busting quacks and standing more broadly for reason and evidence against
the forces of unreason that flourish best in an evidence-free zone. I know you
do this purely because you are passionate about the things David talked about
in his address. As Nick, said, you don’t get anything back – though
50,000 – 70,000 hits is very impressive indeed.
I don’t know whether my frontal lobes are starting to lose their grip – or perhaps it’s my latest detox at the health farm – but I am starting to feel a little more optimistic and have the feeling that the quacks, at least in the UK, are perhaps on the retreat; that this has been a pretty good year for quack-busting. Perhaps it’s a matter of contrast-enhancement because we did start from a pretty low point when in September 2006 the MRHA (Medicines and Healthcare Products Regulatory Agency) gave homoeopathy a boost by licensing its products on the basis not of medical evidence but something called provings. I must admit, I had never heard of provings up till that time and this is what they are.
They’re conducted on volunteers in a reasonable state of health (provers) who generally have no knowledge of what substance they are taking. Doses are repeated until provers start to experience symptoms of a change in state. The provers record everything they experience, whether physical, emotional, mental, or even spiritual, as long as the change in state persists.
At the end of the proving, all the records are collated and compared to find the symptoms, states of mind, feelings and experiences the provers have in common and which can be reasonably attributed to the emerging signature resonance of the substance. Wendy Howard Homoeopathic Practitioner quoted in The Skeptic Express, 2006
It would be an interesting exercise for 6th Formers or even 12 year-olds to be challenged to say why this methodology is flawed. Anyway, the outrageous decision by the MRHA – which seems in this instance to have been influenced by populism rather than by science – galvanised opposition and the annus fairly mirabilis has resulted, with homoeopaths, and herbalists in particular, having a fairly rough time of it.
Many of you will be aware of the campaign fronted by Michael Baum, David Colquhoun, myself and others in partnership with Sense About Science that has had scored some palpable hits against homoeopathy. The campaign to persuade PCTS not to fund homoeopathic treatments on the NHS has attracted huge publicity and, what is more, prompted many PCTs to remove such treatments from their bill of fare. The latest scalp is the withdrawal of funding by the Tunbridge Wells PCT from their local homoeopathic hospital which consequently is likely to be mothballed. There are five Homoeopathic Hospitals in the UK: one down 4 to go.
And then there has been increasing awareness not only of the uselessness of most herbal remedies but, indeed, of their power to do harm. While it is possible that there are some benefits from some herbal remedies – after all, if you fire enough buckshot in the dark, you will sooner or later hit a mosquito’s legs and then say that that was what you were aiming at all the time – there is, as we all know, no evidence that the kind of mixtures dished out by herbalists in their shops do any good. There was huge publicity a couple of weeks ago arising from the study by Edzard Ernst – another person who I am very proud to have as a predecessor in the receipt of this award and who has, I believe, done more than anyone to force CAM to engage with the scientific method – recently published by the Postgraduate Medical Journal. This has provoked widespread demand for properly regulation of these medicines and of those who flog them. Let us hope that this time the MRHA doesn’t fumble the pass.
In short, much grounds for rejoicing. It is as if the nation’s serum Carol Caplan levels are falling and the parts per million of bullshit in the collective consciousness has dropped a notch or two. Now, if this speech was written by a computer, rather than written on a computer, and it had predictive texting, it would next print out the words – ‘There are, however, no grounds for complacency’. Or, as your Chairman, David Bender, no computer he, observed in his report last year – ‘We’, that is to say you, ‘are still needed’. And that’s certainly true. Unreason is always ready to break out, or to return, for reasons that go to the heart of what I shall be talking about this evening and, indeed, what it is about quack medicine that makes it more significant than might appear at first sight. Even as we speak, rejoicing in a year in which in the UK at least quackolysis has definitely had the upper hand over the quacks, herds of bulls are massing, ready to evacuate the contents of their ample colons.
One of the reasons for this is, of course, the way science and non-science are presented to the general public by the media. And the problems are epitomised by what is referred to in the title of my talk: the curse of the media anecdote. Much discussion of medical care – both orthodox and useless – tends to be revolve around, or even to be tethered to, anecdotes.
The media love anecdotes because those who work in them believe (usually correctly) that their customers do. Anecdotes – about a new miracle cure, a drug that is not being made available on the NHS, or the side effects of treatment, or some environmental hazard - sell product: they help to boost circulation, increase audience rating figures, and so on. Data, on the other hand, which take us towards the truth about these things, are less popular.
Anecdotes, however multiplied, do not point the way to reliable knowledge. As the aphorism says, ‘The plural of anecdote is not data’. For an anecdote does not even count as a datum. It may prompt a search for data, as when a chance clinical observation makes someone wonder whether A causes B, or C may have therapeutic benefits in condition D.
These deficiencies count less than they should. For anecdotes, unlike data, have a human face: they are closer to gossip, which is a universal human pastime, and to the things that shape belief formation in everyday life. This is something to which I will return presently. Scientific data - for all that science is one of the most human, as well as the greatest, of human achievements – lack this human face. Their very strength – that they are unattached to persons and personalities – is a PR weakness. At best, data belong to ‘the science community’; worse, to ‘the scientific establishment’; or, worst of all, when they are adopted by, and guide, those who make policy on our behalf, ‘the authorities’.
Let me illustrate the accursed power of the anecdote with one of the most potent examples of recent times: the story told by Jackie Fletcher, the founder of Justice, Awareness and Basic Support or JABS, a group devoted to campaigning on behalf of parents who believe that their children have been damaged by immunisation, most notably by the MMR triple vaccine. Her story is a tragic one. Nearly a decade after the MMR furore broke, it is still news and still being used to influence opinion.
In 2007, The Daily Mail devoted yet another article to her terrible experience, accompanied by a poignant picture of her and her son, ‘who developed autism after being given the MMR jab’:
The change in their bright, loving toddler was sudden. Ten days before his first fit, Robert had been vaccinated against measles, mumps and rubella (MMR). Unlike his older brothers, who had been given the jabs in single doses, Robert had been injected with three viruses at once.
To the Fletchers, the connection seemed obvious..
The Fletchers’ experience is unimaginably terrible: the transformation of a perfectly normal toddler into a child with autism, severe epilepsy, and incontinence, confined to a wheelchair, speaking only the words he knew as a baby, and requiring to be washed and fed. In their case, grief is envenomed with (unjustified) guilt. Mrs Fletcher says, ‘Deep down, I feel that I have caused all Robert’s problems by taking him for the MMR jab and I’m trying to fix it for him and I can’t’.
One would have to have a heart of stone to challenge Mrs Fletcher’s interpretation of what happened; her attempt to make sense of a senseless catastrophe; and her endeavour, through campaigning to protect others’ children against what she sees as a threat, to come to terms with a guilt she feels which, though unwarranted, is real. An anecdote such as this, set among data, is like a queen ant in a colony of drones. Stories have a power that bears no relation to their number or validity. The very fact that those who invoke anecdotes in support of their theories are so convinced by them, adds to their cognitive lustre.
Notwithstanding Nietzsche’s assertion that ‘Convictions are greater enemies of truth than lies’ , certainty, repeatedly asserted, and the anger and grief with which it is charged, increases the impact of the claim embedded in the story, on the reader or listener. When we are invited to assent to an empirical truth in an area in which we have no expertise, we are more inclined to do so if we identify with the person who is purveying it. It is a way of showing solidarity.
The contrast between the media anecdote and the way data are presented in scientific meetings or papers – where there is no reference to the feelings (anger, grief, conviction) of the authors and much concern about sources of error and uncertainty - underlines how disadvantaged the latter are. Of course there are passions behind science – who otherwise could stand the utter tedium of data gathering and learning the techniques of doing so reliably? – but the passions of themselves cut no ice. Indeed, they would be grounds for suspicion from those who are used to what Paul Valéry described as ‘the elementary courtesy of proof’.
Anecdotes also have the ability to attract other anecdotes: when the Fletchers took their stricken child to his many hospital appointments, ‘they encountered other parents who said things like “our child was fine until MMR”’.
This ‘multiplier effect’ is vastly scaled up when the anecdote comes from a celebrity in the media, or is endorsed by such a celebrity. A face that is everywhere is the vehicle for an opinion that is everywhere. The same story is replicated in a million places and has the impact of a million data. In a society where achievement in one area (for example, combining being a moderately competent actor with having an eye-popping figure) makes one an authority in many others, the views of celebrities are the equivalent of the union block vote.
From the point of view of the media, there are secondary benefits of the human story. When data falsifying the Anecdote Based Theory (ABT) start to look overwhelming, other, secondary, anecdotes emerge that nurture the primary one. There is talk of conspiracies, cover-ups by the establishment, and we are reminded of persecution of people (of genius and courageous) who have heretical views.5 The campaigners on behalf of ABT and their supporters look like plucky loners facing a mighty monolithic opposition. Powerful data are discredited simply because they originate from sources that are considered to be on the side of the Big Boys: the scientific establishment, the politicians (who have their own reasons for hiding the truth), and others, such as Big Pharma (who have vested interests). Even the sharing of these data with the public can be presented as an act of oppression. At the height of the MMR controversy, the Department of Health was accused of ‘bullying parents with facts’6.
The power of the anecdote is enhanced by the fact that, once it is in circulation, it looks like an entire half of the overall picture. The media claim that they are required, by their commitment to balance, to give equal hearing to both ABT and science. So Jackie Fletcher talking about her son Robert and giving her untrained critique of the epidemiological data is allowed as much airtime as a scientist bearing news of millions of data obtained under carefully controlled conditions. Given that controversy sells product, maintaining the myth that there are two equal sides to the story is desirable in itself.
The notion of balance is one that should be treated with care, the kind of care that it does not always, or even usually, receive in the media. If applied uncritically, it can result in giving data-free believers in the flatness of the earth the same amount of air-time and column inches as those who support the well-attested notion that the surface of the earth is curved. The Royal Society, quoted in an email by that doughty fighter against junk science, David Colquhoun, has made this point:
When obtaining specialist advice from experts, it is not diversity of opinion that should be balanced, but the weight of opinion accorded to various strands of scientific advice within that diversity. In the name of ‘balance’, the media invariably present opposing views on each side of an argument, regardless of the relative scientific merit of those opinions.
The price of so-called balance that does not take account of the ‘relative scientific merits’ of the opinions being given equal air-time, may be very high indeed. Most people in this room will be aware of Peter Duesberg the maverick molecular biologist who denied the link between AIDs and the human immunovirus. His views were largely discounted in the West and have, thank God have had no influence on science policy, public health policy and those scientific research programmes which have led to the discovery of highly active antiretroviral drugs which have transformed AIDS from a fatal to a chronic disease. Elsewhere, mankind has not been so lucky. Thabo Mbeki’s catastrophic AIDs policies have been hugely influenced by Duesberg’s views and the result is a pyramid of corpses sufficient to block out the sun.
Fortunately, the MMR controversy has a less calamitous outcome. When it was obvious that the link between MMR and autism was unsustainable, and that it never had any basis, some of the media showed signs of contrition. Not all of the press, of course. Melanie Philips still continues her one woman campaign against MMR and combination vaccines that she considers lethal. If her views were listened to, of course, and had any influence on public health policy, the result would be a catastrophe on the scale of what has happened in South Africa. The habit, however, of giving appealing individuals with their moving stories at least as much credence and coverage as unappealing data, of preferring faces to graphs, and vox pops to statistics, does not seem to have been kicked. The recent judgement by the National Institute of Clinical Excellence that certain drugs would not be cost-effective in the treatment of early Alzheimer’s disease on the basis of the evidence so far was dominated by human interest stories.
I found this particularly exasperating because (to declare an interest) I was a member of one of the NICE appraisal committees for three years. This, quite labour-intensive, unremunerated role brought with it two rewards: the knowledge that one was contributing in some small way to ensuring that the finite resources of the NHS would be spent in such a way as to maximise health gain; and the satisfaction, indeed pleasure, in participating in a very high-level, supremely well-informed discussion about the costs and benefits of medicines that took into account basic science, clinical evidence, health economics and social value judgements. Very little of this appeared in the media coverage. The World at One gave extensive air-time to a woman whose father had been given the drugs early in his disease and was ‘convinced’ that he would not have done so well had he been denied them.7
This example illustrates something that is common to much anecdote-based discussion of matters of public interest such as the benefits or harms of medical treatments or the impact of some environmental factor on health: anecdotes privilege the visible over the invisible. In the case of MMR, the millions of children who had been saved from harm by the vaccine were given little foreground; in the case of Alzheimer’s drugs, the impact on all other patients that would result from inappropriate allocation of resources to less cost-effective treatments, was hardly discussed.
I want to step back a little and look beyond their immediate consequences to another reason why the media anecdote is accursed. To think about this I want to rehearse some arguments I have had with people about the use of alternative medicine in the UK. I was recently discussing reflexology and it was pointed out to me that it is pretty harmless and that it doesn’t after all cost much. There was very little evidence that those who used it denied themselves the benefits of orthodox medicine. So why are you getting in such a pet? Well it seems to me that alternative remedies and the theories that are used to support them are a major contribution to the moronisation of the collective consciousness and, at some level, a betrayal of one of the greatest achievements of humanity, and so of humanity itself.
The journey humankind has taken to robust scientific knowledge of the world in which it finds itself has been a long and arduous one. For science has required not only that we somehow penetrate the opacity of nature but also that we overcome the prejudices, the intuitions, the assumptions, the convictions, the dogmas that we ourselves place in the path to reliable, usable knowledge. These obstacles are especially stubborn in medicine, not only because its object is the human body, that from which knowledge takes its rise, but also because its concern is with disease, where emotions, particularly those of hope and despair, tend to have the most powerful influence over our beliefs. The endeavour to cure illness is thus bound to be dogged by deception and self-deception. Hence medicine’s status as a parvenu among the sciences: it is, as Lewis Thomas said, ‘the youngest science’. The greatest miracle of scientific medicine is that medicine became scientific.
The contrast with traditional, or pre-scientific, medicine – the medicine that was all that was available for all but the last 100 years of the 300,000 years of H. sapiens sapiens - is extraordinary in many ways. One of the most important is the fact that scientific medicine, unlike traditional medicine, which is deeply self-satisfied with its knowledge and what it believes to be its effectiveness, is driven by an active uncertainty that is sceptical of received ideas and of authority and is continually seeking to improve on the status quo. It is an irony that scientific medicine is called ‘orthodox’ medicine because, unlike acupuncture, or homeopathy, is constantly scrutinising, challenging, overturning its own beliefs. Like the rest of science, it is in a permanent state of self-critique; and the authority of authorities goes no deeper than the facts that they can invoke. By contrast, acupuncture claims to derive its authority from individuals who laid down the law thousands of years ago. One of the most barefaced examples of proud intellectual stagnation must be the fact that homeopathy has not changed since Samuel Hahnemann offered it as an alternative to the poisoning and butchery that was prevalent at the time. Homeopath still rely on his book, first published in the 1790s, to guide their practice. What he spake, they shall do. There could be no more striking example of frozen orthodoxy and authoritarianism.
Alternative medicine has not progressed to anything approximating clinical science because it has not been prepared to challenge its own authority. So-called orthodox medicine, in an act of collective humility, has cultivated a routine distrust of its own practices. This humility has been almost as important in the development of effective therapies as the discoveries in basic biomedical science. The rejection of anecdote-based medicine in favour of treatments tested in randomised controlled trials and re-tested in meta-analyses etc has accounted for the remarkable advances particularly over the last 50 years, and the quickening pace of increases in life and health expectancy. The humility of shaping clinical practice in accordance with the unmanipulated outcomes of therapeutic trials is also connected with another layer of scepticism: that what looks good in theory (irrespective of how good the theory is) may not benefit patients in the real world.
Perhaps the most remarkable facet of this humility is the willingness of doctors to enter patients (with the latter's consent) into clinical trials run by other clinicians. Submission to the authority of such trials means subordinating one's own personal authority to that of other professionals, many of whom (such as statisticians and biomedical scientists) do not even belong to the medical profession. As Marc Daniels pointed out:
[for clinicians] to be willing to merge their individuality sufficiently to take part in group investigations, to accept only patients approved by an independent team, and to submit results for analysis by an outside investigator involves considerable sacrifice.
The committment to minimising the play of chance, of bias, or of wishful thinking,
is the supreme expression of the self- overcoming that scientific medicine requires.
Avoiding beliefs guided by delusive hope, unfounded authority, superstition
and plain stupidity, it cultivates an attitude of healthy scepticism towards
itself to prevent its practitioners from misleading themselves or their patients.
Its permanent strategy of active uncertainty, and the humility this implies,
is the distinctive virtue of scientific medicine. In the world outside of scientific
medicine, however, humanity has had little time to adjust to this almost inhuman
scrupulousness.
Hence the fact that alternative medicine – and more broadly anecdote-based
treatments - can still live, and even flourish, at least in terms of the number
of customers, and the amount of cash that changes hands, side by side with medicine
that actually makes patients better for reasons that are rooted in a scientific
understanding of the human body and its diseases. Otherwise, it would seem unaccountable
– like an ox-cart at stand in an airport next to a jumbo jet or an Asclepian
temple fitted out with WiFi. Indeed, the constant self-critique of medicine
and the biomedical sciences upon which it is based is often misunderstood. The
self- dissatisfaction and self-transformation - resulting in ever more effective
and, for the most part, less unpleasant, treatments - is seen as a kind of weakness.
A discipline which is marked by a carefully nurtured scepticism towards itself
is sometimes seen as arrogant or in disarray – unlike traditional healers
who are proud never to have had a second thought in 5,000 years.
So this is perhaps the true curse of the media-propagated anecdote and anecdote-based beliefs about medical care: they represent a sickening regression to the cognitively primitive state from which medical science, science, and humanity has fought its way out.
So if we are going to free ourselves from the curse of the media-base anecdote, we need to have a major initiative explaining why anecdotes are misleading, why double-blind randomised controlled trials are necessary – how they work and how, over the last 50 years, they have played such a crucial role in the objectively observed advances in medical care.
A good place to start would be cardiovascular disease, in which randomised controlled trials have had almost as great a role in bringing about improvements in mortality as advances in basic biomedical science. A couple of statistics could introduce the topic. Firstly that cardiovascular disease mortality in men of all ages has fallen in 2005 to 30% of the age specific rate in 1968. Secondly, that deaths from coronary heart disease of people under 65 has fallen by 46% - by nearly a half – in the last ten years. People should be asked to celebrate these facts and to consider the role played by basic science in understanding the processes that lead to cardiovascular disease and methods of preventing or reversing them; and the complementary role of robust clinical trials in ensuring that what should work in theory actually works in practice in the real mess of the real world of doctors and their patients.
Perhaps this is idealistic. The notion that the nation’s serum Daily Mail levels might be reduced by broadsheets offering free tutorials on the methods by which science arrives at robust knowledge and that knowledge is translated into benefits for patients is somewhat utopian. And yet I assume that most journalists (and their sub-editors who write those lethally misleading headlines) want to be able to look back on careers in which they have done more good than harm and advanced, rather than held back, the cause of truth. They are aware that anecdotes may be very effective at conveying the profundity of medical tragedies and the human importance of medical triumphs and disasters. But for making decisions about the absolute and relative values of medical treatments, and more broadly for determining policy, they are of no use whatsoever.
And so I would like to offer a few thoughts, which could be accommodated on a flash card, to help journalists deal responsibly with those anecdotes they and their readers value so much. Before I do so, however, I would like reflect a little more on the power that anecdotes have, which they owe to their central role they play in our making sense of the world in which we find ourselves. And this includes the kind of sense of the mediated world that I am trying to make in this talk – which, it will not have escaped your attention, has itself been rather heavily dependent on anecdotes.
Anecdotes are not an unredeemed evil. As we have already noted, medical science itself has been driven by accidental, observations, anecdotes, though that is merely a prompt to inquiry, the place where it begins, not where it ends. But beyond this, human beings have always told each other stories and storytellers have always commanded a hearing not only because we want to know what happens next but because what is recounted strikes us as standing for something that goes beyond the particular instance recounted.
We imagine beyond the events in the story to a whole class of similar events. ‘This could happen to me’ is extrapolated to ‘This could happen to anyone’; ‘This could happen any time’ to ‘This is happening all the time’. We are taught from earliest childhood to see stories as lessons, as parables: faulty induction is part of our ethical education. Identification with the stories that we are told, seeing them as potentially part of our own biography, is key to the development of that imagination which lies at the heart of sympathy, empathy and hence morality.
The power of stories to fuel extrapolation beyond themselves to an imagined series of general cases is greatly magnified by emotions – as Bertrand Russell pointed out in the passage I have just quoted/ for emotions are the means by which the world we think of ourselves to be surrounded by is illuminated. Angers – generous and mean-spirited – compassion, the sadness that we feel from entering into the particulars of another’s tragedy – these are the great drivers to an extrapolating imagination that takes us beyond what we know of the world to what we suspect of it; supporting that part of our world-picture – by far the greater part – which is not directly vouched for by our own experience. It helps that key figure within the very motor of our cognitive activity, the Professor of Data-Lean Generalisation who helps us to navigate through the world we live in, and to feel that we in some sense cognitively possess it, by being able to command a viewpoint on it. Emotion is the basis of that confirmation bias which is necessary to put together for ourselves a coherent narrative about the world or part of it. This journey, via emotion, from anecdote to world-picture is both our curse and our glory: it lies at the heart of our wider sense of responsibility – the desire to make a positive difference – and, equally, at the root of those generalising hatreds – racism, ‘classism’, all forms of ‘otherism’, which have made the world such an unpleasant place for some and potentially a dangerous place for all. The observation bias that is built into it, and the desire it has to confirm only itself (so that it will be satisfied with very few instances to support a statement of vast generality) enables us to be fluent about things of which we know little; to summarise, in the lounge bar of a pub, the character of an entire race of people in five ill-chosen words.
And it is round about here that we find the true greatness of great science. Like gossip and idle curiosity, it is manifestation of the fundamental human need to make sense of things, to build a picture of the world that vastly exceeds our experience so that we can understand and control it and, so by making the world a safer or more predictable place, to control our destinies. And at the same time, it does not allow the passion to determine what it sees.
It has, in short, managed to elevate its passion for understanding and for reliable knowledge - that will permit what Francis Bacon called ‘the effecting of all things possible’ – above those passions that normally permit us to transcend our limited individual consciousness and fuel our imagination. This is due in no small part to the fact that it is a collective passion that engages all mankind - an average issue of a leading scientific journal will have contributions from scientists from a dozen or more different countries and countless different backgrounds; that it is regulated by rules of inquiry that have been forged in bitter experience and bitter disappointment; and that takes place in the context of institutionalised scepticism.
This separation of passions is lies at the root of the ethic of science and is linked with what Claude Bernard advised as regards imagination: ‘Put off your imagination, as you put off your overcoat, when you enter the laboratory. Put it on again, as you put on your overcoat, when you leave’.
In the end, we must always begin with notions that are insecurely founded on a grossly inadequate database. Kingsley Amis once referred to the way we erect great belief systems on the basis of very little data as ‘the inverted pyramid of piss’ This is perhaps a little harsh, as the necessary starting point for belief formation in both everyday life and in science. In everyday belief formation it tends to be the end-point as well, whereas science moves on from there. While acknowledging the central role of stories, of anecdotes, in awakening us out of our own narrow consciousness to awareness of a world outside, we need equally to acknowledge that an understanding of the world rooted in anecdotes is cognitively underdeveloped. To give anecdotes too great a prominence in the debate on science, and medical science, and science policy, and healthcare policy, is to muddy those waters that it has cost scientist huge effort to clarify.
So, ye journalists, by all means offer your readers anecdotes but do not for one moment deceive either yourselves or your readers into thinking that this is science reporting or that one can move directly from anecdotes to robust generalisable knowledge or recommendations for application of science in healthcare or anywhere else or for the development of health science policy. Here, then, are some reminders that might be useful for journalists who find that they are reporting on science, particularly science as it applies to healthcare:
a) An anecdote is not a scientific datum, though it may justify a study
to acquire data. Do not confuse experience – however intense and harrowing
– with expertise.
b) The journey from a single observation to reliable, generalisable truth is
long and arduous and full of pitfalls. Here are some of those pitfalls (all
illustrated by the MMR furore:
These are hard lessons that mankind has learned reluctantly on the road to useful general truths. Do not be proud of unlearning them out of solidarity with folk common sense. Science in many respects is a triumph over the limitations of interested observers and often over common sense. Ask yourself again and again How typical is the story of the larger picture? How representative are these events? From what population of events are they drawn? For God’s sake give me denominators!
c) When discussing the beneficial or adverse effects of therapies,
or more broadly technologies, consider the whole picture and the whole population.
For example, even a treatment that has an adverse effect in some recipients
may overall have a beneficial effect in the population. The anger of those who
feel the treatment has injured them or their loved ones should prompt review
of the evidence of overall benefit but should not count as evidence in itself.
d) When science fails to support Anecdote-Based Theory, do not assume that expert opinion:
e) When ‘balance’ is sought, it is important that the two views given equal airing should have equal weight. One advocate of ABT, however impassioned, does not deserve as much of a hearing as one million data presented by someone who has acquired them in a way that has survived peer review by experts (who are in many cases professional competitors and on the lookout for errors). The 100% certainty of the anecdote-based theorist should not also be used to make the refusal to claim 100% certainty in science a weakness.
These principles should be known and embraced by all journalists – not just those charged with commenting on science but anyone whose comments may influence the views people may have about science, the application of science, science policy or the larger role of science in society. Indeed, it is outside the columns of science correspondents that the greatest damage to the collective understanding is done. The excellent work by people such as Geoff Watts, Georgina Ferry, Palab Ghosh, Mark Henderson is often overshadowed by the loud and more widely read ignorant certainties of characters such as Melanie Philips. Anyone who is going to make a comment about treatments for breast cancer in the General Features or about the heart attacks in GHQ should, at the very least, have some idea of the approaches to arriving at robust general truths that are employed in science; at the very least they should read Peter Medawar’s Pluto’s Republic, Lewis Wolpert’s The Unnatural Nature of Science, and Theodore Dalrymple’s Mass Listeria.
At any rate, media writers with a conscience should not collude with the cognitively primitive state in which we form most of our everyday beliefs. They should remember Bertrand Russell’s observation that while ‘popular induction depends upon the emotional interest of the instances, not upon their number’, reliable induction depends on numbers not emotional interest. If journalists give excessive weight to individual anecdotes they will trigger unnecessary worry and suffering, and sometimes cause diversion of resource from places where it would be of more use. Vox pop or even vox pop star is not vox dei.
Anecdotes are the first step on the road out of individual subjective consciousness to collective, objective knowledge. They are, however, only the first step. Science has gone far beyond this. If you are going to intervene in a debate on matters already subject to scientific investigation, you need to grow up out of your anecdotage. Otherwise journalism and the anecdotes it loves will get in the way of the collective clarity of thought that is going to be essential if we are to deal with the great challenges ahead of making this planet a safe, and reasonably comfortable, place for those who inhabit it.
The price of putting anecdotes in the driving seat is very high indeed. Uncritical focus on ‘the human story’ at the cost of ignoring or underplaying dull scientific facts may have inhuman consequences, as has been demonstrated by the recent return of measles and the first death from this disease for 14 years in the wake of the unfounded scare over MMR. For this I do not blame poor Jackie Fletcher but those journalists who, out of laziness and perhaps venality, gave her single experience and her heroic but misguided mission, more credence than it deserved.
IN
THE DAYS of old, Eminent Professors (EPs) were the ultimate source of clinical
wisdom. They wrote textbooks approving treatments that, in their experience,
were effective. Usually this advice was repeated in successive editions of their
textbooks, even if new data suggested that some of this advice was not optimal.By 1990 the authority of EPs was being supplanted by Data Revolutionaries (DRs), disrespectful young people who dismissed the experience of EPs as “anecdote”, and instead worshipped a new DataGod. The new Holy Scripture was called “Meta-analysis”, and contained the collected wisdom arising from several clinical trials on each clinical condition. This movement to exterminate EPs and replace anecdote with Meta-analysis in textbooks was supported by HealthWatch, which has the charitable objective to promote valid clinical trials.
At first it appeared that DRs would dominate academic medicine. But now it is becoming clear that the Utopian ideal of totally evidence-based medicine will always be difficult to achieve for several reasons.
First, it is impractical to assemble data from clinical trials of good quality for every possible condition, and for every possible patient. Trials typically compare the effect of the treatment (versus not-treatment as a control group) in adults who are healthy except for the condition being studied. It is useful to show that the treatment being tested is best for this selected group of patients, but it does not mean the treatment is best for real-life patients who are older, or younger, than those in the group tested, or who may also have several other conditions.
Second, good quality clinical trials may require massive resources in money and skilled investigators. The 2004 EU directive on clinical trials has so increased this burden that hardly anyone except pharmaceutical giants can afford to meet the new requirements. This means that there are fewer clinical trials for the DRs to meta-analyse, and they are biased in favour of a drug treatment. There may be a very good treatment involving lifestyle changes that never appears in a meta-analysis, because no non-profit sponsor could afford to finance the trial.
Third, editors of peer-reviewed journals want to fill their pages with information that subscribers want to read. It is generally more interesting to read that a treatment works than that it does not. Therefore trials with a negative outcome are less likely to be published either because the editor rejected them or, more often, because the researchers did not write them up and submit them.
Fourth, researchers and DRs (and perhaps even editors of journals!) are not always pure and impartial scientists—they may well have conflicts of interest. The solution to this problem is to have Eminent Editors (EE) who are pure, impartial and ever vigilant to present (as far as possible) a perfect unbiased summary of the evidence of efficacy for every treatment of every condition. So we can all have confidence in the purity of the EE he (or she) must be sponsored by an organisation of irreproachable integrity, and the methods used to reach conclusions must be totally transparent, so that sceptical readers can replicate the process.
Perhaps the nearest we can get to an EE is Dr David I Tovey, editorial director for the British Medical Journal’s Knowledge department, which produces evidence-based products for clinicians and the public. It is therefore worrying that on the BMJ Clinical Evidence website [1] a statement on the current knowledge about effectiveness of treatments suggests the meticulous rules for meta-analyses are not being obeyed:
“Of around 2500 treatments covered 13% are rated as beneficial, 23% likely to be beneficial, 8% as trade off between benefits and harms, 6% unlikely to be beneficial, 4% likely to be ineffective or harmful, and 46%, the largest proportion, of unknown effectiveness.”
Can it be true? The selection of 240 common conditions is arbitrary, but they are listed, so I can replicate that. But when I try to replicate the list of treatments I am baffled [2]. To claim that “46% of around 2500 treatments are of unknown effectiveness” (my italics) makes no sense unless the criteria for selecting which treatments to include and which to leave out are clearly stated. Some people may treat their wrinkles by washing their faces in the dew on May mornings. If the treatment list included that, and all the other innumerable evidence-free folk remedies for every common condition, the treatment list would have thousands of entries of which the percentage “unknown” would approach 100%.
On the other hand the list of treatments analysed could have been limited to those which had been tested by good randomised trials, in which case the “unknown effectiveness” category would shrink to nearly 0%. Without the selection criteria specified, the figure of “46%” is meaningless. Worse, it is even snatched up with glee by the alternative health lobby, who have used it to claim that nearly half of orthodox treatments are of unknown effectiveness [3], when in reality many of the treatments analysed are themselves alternative remedies (and are, not surprisingly, over-represented in the “unknown effectiveness” category).
So, is Dr Tovey really committed to the crucial value of evidence? In a letter
to the BMJ he wrote: “A principle of evidence-based practice is that the
evidence should be only one influence on clinical decision making, alongside
the expertise and perspectives of both patients and clinicians. However uncomfortable
for health system planners, an evidence based service should reflect expressed
patient preference.” [4]
In other words, now the Eminent Professor suggests that “expertise and
perspectives of both patients and clinicians” (what we used to call anecdote)
should have a place on the rostrum along with meta-analyses of clinical trials.
Maybe that will bring some benefits, but if it is a sign that he wishes to hark
back to the days where anecdote supplanted evidence I think Dr Tovey should
reconsider his position as editorial director of Clinical Knowledge.
John Garrow
Emeritus Professor of Human Nutrition, University of London
1. http://clinicalevidence.bmj.com/ceweb/about/knowledge.jsp
2. Garrow JS. How much of orthodox medicine is evidence based? BMJ 2007; 335:
951 (10th November).
3. http://www.newscientist.com/article/mg19125630.200-happy-homeopaths.html
4. Tovey DI. Evidence should not be the only consideration. BMJ 2007; 335: 952
(10th November).
Consultant
obstetrician Susan Bewley wrote to express her fury at an article which appeared
in the last issue:Dear Healthwatch Editor,
I note the article entitled “HRT under fire––again” by Michael Henk (retired) in the latest Healthwatch Newsletter (issue 67, October 2007). He writes about a newspaper article by Margaret Cook [1] (haematologist) in the Daily Telegraph that was based on an editorial [2] in a well respected international journal (the Lancet) written by a well respected author (Valerie Beral) about a potentially important topic (whether HRT causes ovarian cancer). I’m not going to get into the rights and wrongs of a complex topic except to say that I was appalled this was accepted for publication:
Surely HealthWatch readers deserve better?
SUSAN BEWLEY
1. HRT: kill or cure? Cook, M. The Daily Telegraph; 20 April 2007. (Article
no longer available online).
2. Beral V. Ovarian cancer and hormone replacement therapy in the Million Women
Study. Lancet 2007; 369: 1703–10.
Dear HealthWatch Editor,
James May (HealthWatch Newsletter
issue 67, October 2007) is rather too kind to post-modernist views on knowledge:
the idea that no understanding of the material world can be relied upon more
than any other is not merely “gullible and naive”—it is also,
frankly, rather childish.
The reason why conventional science is preferable to what is taught in Hogwarts
is that, by and large, science works and Harry Potter-type magic doesn’t.
Does James May’s interlocutor live in a house with electricity, water
and mains drainage? Does she travel by any form of motorised transport? Does
she use sun cream when lying out in the garden in the summer? If she does, she
is using technologies derived from conventional science, because they work.
The conventions, classifications and conclusions used in such science and technology
are not arbitrary, like how you might classify pieces of flotsam and jetsam—they
are derived from a long process of observation, experiment, prediction and further
experiment.
ROGER FISKEN
Dr Fisken’s views are echoed by James May’s fellow HealthWatch committee
member Les Rose, who answers in “last word” below. But will it be
the last word? If you want to join the debate, log onto www.healthwatch-uk.org
and join the discussion forum by clicking “members forum” (last
button on the left-hand panel)
James
May is a bit too tolerant of post-modernism (see his article in the HealthWatch
Newsletter issue 67, October 2007), although I do see what he is trying
to explain. One of the main reasons that science is so often mistrusted is that
it is in the business of reducing uncertainty, and less concerned with providing
the “yes” or “no” answers that many non-scientists demand.
We have statistics so that we can assess this uncertainty surrounding a piece of evidence. Thus I really do not see the merit in failing to recognise the difference between a claim which lacks convincing evidence, because of high uncertainty, and one for which the uncertainty is low––as post-modernists seem to do. Neither do I see the utility of living one’s life like this. By accepting a claim which has good evidence to support it, I run a low risk of failure. By refusing all claims, however strong the evidence, none of my decisions will have any value.
Science is also the business of the ‘crash testing of ideas’. Everything we claim is open to challenge, and it is good for the scientific method itself to be challenged. It is true that some modernist scientists have fallen into the trap of proclaiming absolute empirical truths (James’ “failed epistemology”), but this doesn’t justify the abandonment of the search for truth, or the denial of evidence.
Thus by explaining this continuum of uncertainty, scientists can very easily demolish the pretentious fantasies of the post-modernists. This is not to say that some facts do not approach the certainty that many non-scientists expect. To all intents and purposes the Earth’s orbit around the Sun is an absolute truth, and without assuming that it is, space exploration would be impossible. Things are rarely so certain in medicine.
Am I reinventing James May’s “critical realism” with this reliance on the role of uncertainty? Even if I am, I don’t see the concept as new, and I retain a modicum of sympathy for those modernist scientists who have nailed their colours to a mast of pragmatism (at least, those who didn’t get their empirical facts wrong!). It is not a failure to strive for objective evidence, while accepting that there may be uncertainty as to its degree of objectivity. This is why scientists use controls in experiments. It is a failure to deny the value of all evidence. Let’s be clear as to where the responsibility lies: the onus is on those who make claims, to provide supporting evidence. The role of sceptics is to challenge claims, and to evaluate the evidence. If it looks good enough, we’ll accept it, but always with the qualification that later evidence might overturn it. There is nothing noble or admirable about never accepting the evidence. Neither is it clever to denigrate those who devote their lives to the quest for knowledge. As Sokal and colleagues demonstrated, post-modernists can’t even recognise falsehood when it is thrust in front of them.
Les Rose
Freelance consultant clinical scientist and
HealthWatch committee member
The Conservative Party may not have done their homework before launching their new committee. Plans for an Independent NHS board with responsibility for allocating the £90 billion a year health service budget and setting national standards of care were unveiled by Andrew Lansley, the shadow home secretary, in November.
Rather like HealthWatch, this board would seek to influence decisions in the
NHS by promoting clinical trials and evidence-based medicine, rooting out treatments
(both orthodox and complementary) whose benefits are unproven, and therefore
saving the country from unnecessary expenditure.
If patients felt the NHS failed to deliver care to the defined standard, they
could complain to a completely new body which would gauge patients’ opinions
and ensure their views were acted upon.