The need for evidence-based medicine
At the Annual General Meeting of Health Watch at Regent College, London this year we were delighted to welcome guest speaker Professor David Sackett from the Oxford Radcliffe Hospital. The professor's compelling talk described his team's important work on the need for accurate and up-to-date information in making clinical decisions.
Doctors need new clinical information up to 100 times a week but are unlikely to receive it because their textbooks are out of date, and their journals disorganised. Yet this dearth of accurate information could influence eight important decisions every day. So began Professor David Sackett, visiting professor at the University of Oxford, when he addressed the 7th Annual General Meeting of HealthWatch on 23rd October 1995.
By questioning a group of general practitioners while they saw their patients, Professor Sackett's team identified up to 16 needs for new, clinically-important information in just half a day, of which a quarter were related to diagnosis.
However, only 30% of these information needs were met in the doctor's workplace, and most of these by asking colleagues.
The literature shows that doctors' ability to obtain the information they need is limited by out-of-date textbooks, journals that are too disorganised to be useful, and by simply not having enough time to read. Even the keenest GPs spend only one hour a week reading the medical literature, and this is clearly not enough to keep pace with therapeutic advances.
The net effect of this unfulfilled need for important new information is that it can lead to progressive declines in clinical competency. It has been shown repeatedly that there is a negative correlation between a doctors' knowledge of up to date care and the years that have elapsed since completing formal training. For example, in one study of clinical behaviour, the decision to start antihypertensive drugs was more closely linked to the number of years since medical school graduation in the doctor than to the severity of target organ damage in the patient.
It is clear, said Professor Sackett, that doctors need far readier access to clinically-important information. The problem is to distill the message buried in some 600,000 published randomised controlled trials into a form which was accessible to clinicians when they needed it, and on which they would be able to base their treatment decisions.
Critics of conventional medicine, said the professor, have claimed that fewer than 15% of medical interventions are supported by solid scientific evidence, leaving between 80 and 90% in the realms of quackery. A more reassuring picture emerged when the professor’s team reviewed the diagnoses made and interventions performed upon 121 patients admitted during April 1995 to the John Radcliffe Hospital.
He found that the vast majority - 82% - of interventions were justified on the basis of evidence-based medicine, and in 53% of all cases that evidence came from randomised controlled trials. In only 18% of cases were treatments were performed for which there was not substantial clinical evidence, and Professor Sackett found that these involved mainly non-acute disorders for which there are no known effective treatments.
The professor went on to outline how clinicians can practice evidence-based medicine for themselves as a life-long, self-directed learning process. The discipline involves:
- converting information needs into answerable questions;
- tracking down efficiently the best evidence with which to answer them;
- critically appraising that evidence for its validity and usefulness;
- applying the results and evaluating performance.
Research in Canada has shown that when equipped with these skills, graduates not only made more accurate diagnostic and management decisions, but they retained a high level of clinical competence and stayed more up to date than their traditionally trained peers as long as 15 years after graduation.
After his talk at the HealthWatch 7th Annual General Meeting Mr Nick Ross announced that Professor David Sackett was the unanimous choice of the committee for the Third Annual HealthWatch Award. Professor Sackett was presented with a handsome silver-plate comport-a dish for dessert, raised on a stem-inscribed with these words:
To Professor David Sackett in recognition of his distinguished contributions to medical research, education and the reliable assessment of treatment.
Medical journalism: triumphs and pitfalls
Geoff Watts, presenter of Radio Four's Medicine Now, is the recipient of the first annual HealthWatch Award. Here is a shortened version of the talk he gave to HealthWatch members.
The medium I work in - radio - has strengths and weaknesses. TV relies on pictures: the words are subsidiary. Print journalism does rely on words but they are mostly those of the journalists themselves, fleshed out with quotes from people whose views and ideas are being reported, disputed or analysed. But in radio this 'fleshing out with quotes' actually forms the greater part of what is broadcast. This is both the strength and the weakness of radio journalism.
It is a strength because radio journalists have the authority that comes from using a contributor's own words and voice. It also reduces the likelihood of distortion.
It is a weakness because radio journalists are more reliant on the fluency and articulacy of their interviewees. This can be a problem when dealing with complicated technical matters.
Broadcaster vs. medic
Doctors can't ignore their patients' prejudices or advocate therapies that patients don't want. Nor can broadcasters survive by making programmes that people don't want to watch or listen to. Compromises have to be made by both.
But there is a fundamental difference: doctors don't have to entertain their audiences, which are largely captive. The media do have to entertain -using that word in the sense of 'amuse and intrigue': to hold the attention.
One result is that the media do not necessarily emphasise what doctors think are the most important issues or angles. The 'responsible' journalist tempers the presentation of pure truth with enough entertainment to woo the audience; the less responsible provides nothing but entertainment - from sob stories to horror stories. But, when criticising the serious media, recall that they too have to make themselves attractive to audiences. No audience means, ultimately, no programme.
Doctors and broadcasters can fall out over the question of language, particularly when talking about matters that are technically difficult. Doctors are generally better than scientists at speaking in plain English but there is the danger of patronising the audience with talk of 'germs' and 'tummies.
Are doctors and scientists afraid to talk in ordinary language? Do they fear they won't sound important enough? Some can't seem to get away from the idea that they're talking to their peers. They're not. To talk simply and directly is not to trivialise.
Disasters and triumphs
Developments that would have been impossible without the cooperation of the media include the reduction in cot death following changes in the recommended sleeping position and the spread of knowledge about safe sex. People are now better informed about health.
Things go wrong for many reasons. There is genuine misunderstanding of the facts. Sometimes those providing the facts don't help. There is conflicting propaganda, for example, over the role of animal fats in heart disease.
Journalists occasionally decide their conclusions before they write the story. One TV production company and one particular journalist seek to champion the unorthodox ideas of Stanford University's Peter Duesberg about HIV. Provided no violence is done to the truth, and the competing claims are given a fair hearing, a partisan or unorthodox line of argument presents no problems - but it must be declared as such.
Anecdote is legitimately the basis of much journalism - to rule it out would be absurd. But, unaccompanied by the examination of more objective evidence it results in uncritical coverage of miracle cures and gee-whizz technology and a wide-eyed enthusiasm for every dubious idea dreamed up on the fringe from total allergy syndrome to the health benefits of sitting inside pyramids. It's an area in which journalists' natural scepticism can fall prey to the lure of a good story -and is a regular source of pitfalls.
Changes of points of view inside the medical establishment can lead to stories going wrong. The value of dietary fibre, the health benefits of moderate drinking and the violence induced by lowering blood cholesterol are just a few of the issues on which the research establishment has been, or is in the process of, rethinking its ideas. Changes of understanding can't be avoided - but they do create problems.
Publicity-seeking by individual doctors can generate bogus, dubious or non-stories. Motives range from self-agrandisement to bringing in private work to shroud-waving for an institution.
The practice of medicine is nothing like as international as the science on which it is (supposed to be) based. There are transatlantic differences which can be subtle and create further pitfalls for the unwary.
Finally, should medical reporters working for the lay media try to behave like editors of peer-reviewed journals? This is not practicable or desirable. The periodic failures of journalism are not so much a problem to be solved as a (sometimes unavoidable) phenomenon to be monitored, noted and kept within bounds of acceptability.
Pitfalls of medical journalism
Leading medical journalist Annabel Ferriman, winner of the 1997 HealthWatch Award, entertained members at HealthWatch's ninth Annual General Meeting in October when she explained the factors which can conspire against media reports being quite as accurate as health professionals would like them to be. Her talk is summarised here.
Pitfalls include the pressure to produce, the need to be first, the search for simplicity, the media's dislike of neutrality and commercial and political pressures. I'm going to confess that I have fallen into all of them.
The Pressure to Produce
Newspapers are not universities. They don't pay journalists to research into medical matters, fully inform themselves and then occasionally impart some of their wisdom to the British public. One day a journalist is a general reporter; the next, he or she is medical correspondent. From that day onwards, she's expected to under-stand the most complex issues and translate them into comprehensible language for the lay reader. Moreover; most of the time points are won only for disclosing information and producing copy, not for deciding that a story should not be published because it's without foundation or overblown. Experienced medical journalists who've seen dozens of breakthroughs and scares and who suggest to their news editor that the latest is not worth covering, are likely to be branded negative or stale or lazy.
For me and others who work on weeklies, or who are feature writers rather than daily journalists, the problem comes when you sell an idea to the news editor or features editor that you then cannot "stand up". When you sell it, you have usually only half-researched it and often, on further research, you discover your first impressions were wrong.
An example: I sold the idea to the medical editor of The Independent that it was a scandal that men were not being screened for prostate cancer; but discovered on further research that the issue was rather more complex. I could hardly then sell the idea that it was a good thing that we are not screening everyone for it, because that is a non-story.
This is a well-known phenomenon in journalism known as the 'one 'phone call too many'. That is, you've made a 'phone call that has knocked down your whole story. What do you do? Suppress the knowledge and pretend you never made it? Sometimes that happens. Or include the qualifying statements in paragraph 24, so that it's not too obvious that your whole story is somewhat flimsy.
The need to be first
This can result in dangerous half-truths. I have also been guilty of this: witness my piece on discovering the gene for schizophrenia. A chap at the Middlesex thought he had done so, I wrote it up and, as quite often happens, it was a false dawn. The search for simplicity.
When I first started in medical journalism, I was told by Michael O'Donnell that there were only three medical stories: the major breakthrough, the major scare and the major scandal.
The search for simplicity (usually based on the assumption that your reader has a 50-second attention span) often means that complicated stories are over-simplified to the point of nonsense.
But this need for simplicity has another side to it. Editors and readers love nothing better than the story which suggests a lot of clever scientists have been working away for years to discover the key to a healthy life; or a cure for cancer; or the answer to multiple sclerosis. And then along comes a patient, or alternative medicine practitioner, who discovers that all that scientific research was quite unnecessary, that medics had been overcomplicating things and the answer was quite simple. The media's dislike of neutrality.
Newspapers and television producers love to name the guilty men. Doctors, scientists and drug companies are often cast as the villains; the poor long-suffering patient as the hero or heroine. News editors don't tend to like stories that say: on the one hand this and on the other hand that. I was quite good at those, but they were usually put on page 10, so I didn't get many points for that.
I recently discovered that the Hammersmith were planning to offer pre-implantation diagnosis for couples carrying the breast cancer gene. I tried to write a completely neutral story, but it ended up with the headline, 'Breast cancer embryos may be culled'.
Commercial and time pressures
The fact is that most journalists are inundated with press releases every day, pushing this or that product, drug, cure, book or message. It is very easy, if you are up against a deadline, simply to reproduce a press release that you have been sent, without giving it the scrutiny that you should. I have seen press releases reproduced almost verbatim by journalists, for example a particularly idiosyncratic introduction to a Department of Health release on skin cancer was recently reproduced word for word in the Guardian. In this case, not many people would find it sinister. But it shows how easy it can be to manipulate the press.
Some PR firms spoon-feed journalists and are rewarded by column inches. I had to produce a column very quickly for the Telegraph recently, and there happened to land on my desk a long press briefing about bedwetting. It even had the magic words, 'We have a case study which could be of some interest.' I am slightly ashamed to say that I used a great deal that was in the briefing, inter-viewed the case study and banged out 1,200 words in no time. My only defence is that I had very little time, and that I did include a lot of advice about behavioural ways of tackling bedwetting, as well as the fact that there is now a good pill available.
In the light of the strong pressures working against good medical journalism, it is something of a miracle that medical coverage is as good as it is.
Leading journalist Polly Toynbee was the winner of the 1998 HealthWatch Award for her outstanding contributions in informing the public throughout her career. Ms Toynbee was unable to accept the award in person at this year's Annual General Meeting, but she prepared for us the following report of her views on issues that concern HealthWatch, ranging from health screening to bogus diagnostic techniques.
Questions were put to Ms Toynbee on HealthWatch's behalf by Chairman Professor John Garrow. He began by asking whether she believed that screening programmes involving apparently healthy people do more good than harm? She replied, "As a member of the National Screening Committee, we are bombarded with requests for national screening programmes for a huge variety of ailments, some of them utterly esoteric and rare".
There are currently several hundred haphazard screening programmes in operation, done differently or not at all in different areas, many of them of deeply dubious quality or purpose, many quite expensive. It is plainly going to take time and a lot of persuasion from the centre to get people to stop doing what they have done for a long time, where there are no clear beneficial outcomes. It is much harder to stop existing screening schemes than it is to start new ones.
We started out two years ago and have now drawn up a protocol with which to judge any proposed - or existing - screening programme.
The key principles are that the screening should be for a relatively common disease, that the process should be easy to carry out, will gain public support, a high quality should be maintainable, and that there should be a clear remedy with a definite health gain if disease is detected. The test should provide an acceptable level of false positives and negatives, and people should understand that there will always be some false positives and negatives. Every programme should have a strict quality assurance and effectiveness evaluation built into it. Cost is one element in the consideration, but it is by no means the overriding one. One of the first requests we dealt with was for prostate cancer screening. A huge head of medical pressure had built up for this procedure amongst urologists. There was also quite a strong political pressure from the small but vocal nascent men's movement who felt men should have the same access to screening as women did for breast and cervical cancer. This was the first real test of our resolve to stick by the principles we had drawn up, for prostate screening failed on several counts. The most important of these was that it was still unclear whether if prostate cancer is detected, surgical intervention makes any difference to the outcome. Prostate cancer is found in a high number of older men who will outlive their disease, and surgery can damage their quality of life without offering a clear health gain.
On the other hand, we have decided to go ahead with pilot schemes to test out the effectiveness of screening for bowel cancer. Catching it early has very clear health gains in saving lives, but the pilot will show what level of compliance it gains from the public who have to take part in a test many of the squeamish may find unacceptable. In my view, the work of the committee is very thorough, properly analytical, and well aware that the anxiety which can be caused in screening programmes - and the money wasted - can end up causing the individual and the NHS more harm than good. There is a strong push for screening for every disorder, so we regard it as most important to try to educate the medical profession and the public on the principles of screening. Our work is as public as possible and all papers are available on our own web site on the Internet.
Unvalidated methods of healthcare in conventional/alternative medicine. Is this a great evil to be rooted out? If so, how? Or should we apply caveat emptor?
Personally, I am appalled at the galloping growth in belief in 'alternative' health. Alternative only means anything which hasn't been properly subjected to clinical trials. Prince Charles keeps trying to bring together orthodox and unorthodox methods. Newspapers that thirty years ago would have had no truck with this now peddle it daily on their health pages - especially the Daily Mail - alongside articles on real health care. It is strange and alarming to me that just as more people are being educated for longer, many more doing sciences at least until the age of 16, the world seems eager to abandon basic scientific methods in favour of alchemical mumbo jumbo.
As a first step, there should be a strict edict within the NHS that no unvalidated method should ever be part of the health service. (This might not include osteopathy: for all its weird underlying beliefs, there is good evidence of its efficacy, but only in the first six weeks of back pain - not for chronic long - term conditions.) If the NHS, from the department, took a firm and public stand on this, we would at least start the debate. The Department for Education should take the same view where it is creeping into university courses - usually new universities - often with sponsorship from the makers of herbal and homoeopathic remedies.
I don't think 'caveat emptor' is enough here, when so many newspapers and magazines tell people unscientific stories about miracle cures. Perhaps all remedies, and peddlers of them, should have to tell patients clearly that their quackery has not been tested scientifically. After all, some agents such as ginseng (though goodness knows why only one or two) are specifically banned under Advertising Standards rules from claiming any health - giving properties whatever. Alas, people still take it!
When a procedure is shown to be completely bogus (e.g. iridology or applied kinesiology as a method of diagnosis) this hardly affects the popular demand for it. Why is this so? Does it matter?
Why do people cling to bogus procedures? Because they want to believe in magic and miracles. And many hate and fear doctors for phobic reasons. Some people who are well want to think themselves ill, or that they are suffering from interesting allergies, to shore up their view of themselves as more interesting people. Or simply because they are emotionally in need of feeling someone is caring for them. The one bit of good the alternative practitioners may do is to take troublesome patients off the hands of hard - pressed doctors. But obviously that has risks, since some of these patients may be ill with real diseases not at first detected.
Is there inevitably a conflict between a scientific approach to medicine and a loss of the holistic approach? Does it matter?
I don't think there's any reason for the holistic idea to be exclusive to alternative practice. We all know doctors who are good scientists and horrible to patients, and vice versa. Increasingly patients won't tolerate rude and unkind doctors, and tend to walk with their feet, if they can. Amongst younger doctors these days I'm much encouraged to hear them talk a great deal about the good they can do by treating people well, listening to them and treating them as a whole person, not as a body-part.
Clearly medical schools have become far more aware of this aspect of treatment. Also, increasing use of counsellors within GP practices can do much to help.
Will evidence - based medicine destroy the efficacy of placebo treatment in situations (e.g. multiple sclerosis) in which a good placebo is the best treatment on offer?
Evidence-based medicine is the only way forward and the government's new National Institute for Clinical Effectiveness ought to do much to weed out treatments that don't work and encourage those that do. It ought, in the end, to give people more, not less, confidence in their doctors.
The placebo effect is very powerful, and good doctors know how to use it to suit each of their patients. Some patients are scientifically minded, others just want to trust their doctor blindly. Oddly, in my experience, this has nothing to do with intelligence or education, but is an attitude of mind people adopt when faced with serious illness. Some rush out to read every book, others prefer trust and hope, or simply mentally avoid it altogether. But once we elevate the placebo effect above science, all is lost.
The GM Furore: Who's the Blame?
Dr Bernard Dixon, science writer and former editor of New Scientist magazine, is the winner of this year's HealthWatch award for his many years of work in providing high quality information on scientific issues. After receiving his award at the HealthWatch Annual General Meeting in October, Dr Dixon gave HealthWatch members an illuminating and most enjoyable talk on what is possibly the most discussed yet least understood issue currently in the news.
It would be oversimplifying matters to blame the furore surrounding genetically modified foods entirely on the media. But they, along with many others, have played their part. Genetic modification is not new. The techniques were being developed twenty-five years ago when scientists voiced their concerns about possible dangers at a meeting at Pacific Grove, California. These were famously reported on in the American magazine Rolling Stone in an article entitled "Pandora's Box".
But the hysteria we have seen recently is a uniquely British phenomenon. And it was not until 10 August, 1998 that it was essentially triggered by a World in Action television programme. This described experiments in which Arpad Pusztai of the Rowett Research Institute, Aberdeen, had apparently demonstrated that mice developed stunted growth and an impaired immune response as a result of eating genetically altered raw potatoes. It seemed that Pusztai had inserted into the potatoes a gene coding for a lectin (a type of protein produced by many plants as natural insecticides). Amplified by banner headlines in newspapers and through television newscasts worldwide, World in Action's theme was the horror of "Frankenstein food".
There were, however, immediate grounds for caution over the practical significance of the Aberdeen work. Firstly, the results on potatoes were preliminary and unpublished. Secondly, even if the very worst interpretation were placed on this research, it scarcely justified the condemning of all GM food. However, the media claque which both preceded and followed the TV programme was not cautious. The Express ran a front-page splash, "Genetic crops stunt growth", accompanied by an editorial on Frankenstein food ("The latest revelations... raise the prospect that scientists might be creating something truly dreadful"). The Daily Mail's front-page story announced that the discovery undermined repeated assurances from manufacturers and governments that such foods posed no risks.
The furore quickly ended in farce. Barely three days after World in Action released its bombshell, Rowett Director Philip James announced that the experiments had not been done using genetic methodology at all, but by spiking potatoes with a lectin. Arpad Pusztai had been suspended.
The Daily Mail, to its credit, gave the same prominence to a front-page announcement that "Food scientist got it wrong" as to the original story. Other papers performed less creditably.
One clear lesson, at this stage, was that, just as scientists should not release data to the media until they have withstood appropriate critical scrutiny, so journalists need to question whether and where research claims have been published. While the refereeing process cannot guarantee absolute veracity, it undoubtedly helps to minimise error in the professional domain and needless alarm in the wider public forum.
Secondly, people commenting on genetic manipulation need to recognise that adverse effects coming to light during screening tests confirm, rather than repudiate, the effectiveness of those procedures.
With rare exceptions, this was not an impressive episode in the media coverage of science.
But then, six months later, it all erupted again. "Frankenstein food fiasco", "Shops in fear over GM food" and "Safety fears at 70 sites testing GM crops" were typical headlines on 12 February 1999. The trigger was a Guardian report that 20 scientists had supported Arpad Pusztai in stating that mice suffered adverse effects when fed on raw potatoes containing a lectin gene. Unfortunately, neither The Guardian article nor a press conference later in the day revealed precisely what Pusztai had done. Speakers said they had written a report, but it was unavailable. And-six months after the original claim-the work remained unpublished.
Such niceties did not deter the many reporters (not science correspondents) who went into overdrive in the ensuing days. "Scientists back findings of ousted expert" announced The Daily Telegraph. "Scientists are vying to produce the ultimate in Frankenstein foods-plants and animals with human genes", added The Express two days later Then The Guardian published a list of "GM foods to avoid like the plague", with names of companies, brands and products.
Absent from all of this was any recognition that the term "GM food" had three very different meanings. Not one writer, over several days, explained that a cheese, sugar or oil made by a recombinant organism differs considerably from a product such as tomato puree containing denatured DNA, and in turn from a plant containing viable genes.
Virtually no attention was given to the laborious vetting procedures of the Advisory Committee on Novel Foods and Processes. Also ignored were the committee chairman's public request for the potato/lectin results, her criticism of the way information had been released directly to the press, the Rowett Institute's rejection of the new claims as misleading and its call for open publication of the findings.
Instead of addressing reality, most otherwise serious programmes and publications took the simpler course of accelerating the bandwagon. The Sunday Times announced that "GM food is already widely available. Now scientists warn it could be a health risk"-suggesting that tomato puree or vegetarian cheese could be as dangerous to eat as raw, poisoned potatoes.
Such antics perplexed many scientists outside the UK (and many inside). Yet there were wider messages. One was the danger that national hysteria could not only jeopardise an entire industry in one country but create tidal waves elsewhere and indeed threaten international trade.
It is tempting to blame the media solely for Britain's GM food furore of 1998-9: there is much incriminating evidence. Yet others played significant roles, sometimes unwittingly. The furore was ignited not by a journalist but by a scientist, Arpad Pusztai. It was then supercharged by the group of 20 scientists in February 1999 whose protest, it later emerged, had been co-ordinated by Friends of the Earth.
Consider too the British Medical Association's report on The impact of Genetic Modification on Agriculture, Food and Health, issued in May 1999, and the ensuing media coverage-"Doctors on alert for GM diseases" (The Times) and "Doctors sound alarm on GM food" (The Independent).
But what did the BMA report actually say? One of its key conclusions was that "transgenic products may adversely affect people suffering from allergies. Soyabean containing genetic material from Brazil nuts cause reactions in individuals allergic to nuts". In fact, the single reference which the BMA used to back its claims was to a paper showing that an allergen from a food already known to be allergenic could be transferred into another food by genetic engineering. In other words, a screening test on a well recognised allergen, carried out specifically to exclude hazards of this sort, has been transformed in the public mind into the threat of unforeseen allergies lurking in our food. Neither the BMA report, nor any of the reports on the report, pointed out that one of the most valuable potential applications of genetic modification to food is to remove possible allergens by deleting the appropriate genes.
A quite different but nevertheless significant factor was the growing realisation that, with the increasing commercialisation of Britain's university science departments and publicly funded research institutes in recent years, it was now very difficult to find truly independent expertise for the evaluation of contentious issues such as the alleged risks associated with genetic modification.
Campaigning groups such as Greenpeace, Friends of the Earth and the Soil Association clearly played major roles too. (They might have been told that one of the principal motives for genetically modifying crop plants was to give them inbuilt resistance to attack by pests. This is far preferable to using chemical insecticides, and indeed is the very style of "biological control" which Rachel Carson advocated in Silent Spring in 1963.) So (inadvertently) did certain scientists who went over the top in dismissing public concerns as simply irrational.
Another significant voice was that of Prince Charles, who on several occasions spoke out against GM foods. A final, powerful influence was a circulation war between Britain's national newspapers, which no doubt helped to increase the temperature of sensationalism which characterised much of the media coverage.
The most regrettable feature of the UK furore over GM foods has been the pervasive insinuation that science in general is not positive but negative. Of course, the development of genetic modification, like every other discipline, will probably be accompanied by some risks-though none have come to light since the advent of recombinant DNA over a quarter of a century ago-and mistakes will be made. But given the practical fruits of scientific research in healthcare, agriculture, environmental protection and other fields, the idea that science simply creates problems which it cannot contain is absurd.
Some material adapted from: Dixon, B. What are science journalists for? Information Services & Use 1999; 19: 75-81.