Poking your nose in where it’s not wanted: the dark side of investigating healthcare
The 25th HealthWatch Award was presented to the former BMJ Investigations Editor Deborah Cohen, in recognition of her courageous reporting of medical issues in the face of attack from vested interests. She received her award from HealthWatch president Nick Ross at the 2017 HealthWatch AGM on Tuesday 17 October 2017, and the following text is adapted from her presentation.
It has meant a lot to me to know that others recognise that you’re reporting in the face of vitriol and attacks from vested interests.
The first big story I did was in 2009. HealthWatch members will already be familiar with the Tamiflu story and how the Cochrane Collaboration struggled to access clinical trial data for their review of this flu drug. The investigation into the missing data ended in my in-tray. I have degrees in medicine and medical journalism, so it was something to get my teeth into, and it turned out to be an interesting journey. As you’ll recall, the lead authors of the clinical trials had told us they didn’t have access to the primary data. The manufacturers, Roche, gave us observational data. But not the clinical trial data.
One of the privileges of working at the BMJ is you have access to some really good academics who will drop everything to help you out – they know who they are. So we set them to work making sense of this observational data that we had. Roche maintained that Tamiflu not only reduces the incidence of secondary complications by 67%, but it also reduced the rate of hospital admissions by 61%. But when we re-analyzed the observational data we found it didn’t say that. We published our report,(1) and the allegations we made were quite controversial. No satisfactory explanations had been given for the missing clinical trial data. Yet there were journalists who’d been covering the flu pandemic for a while, who’d taken it at face value that Tamiflu must be effective enough to merit the UK government stockpiling it – at a cost of £424m between 2006 and 2013.
Conflicts and controversies
In the course of our search we came across whistleblowers who had written up the clinical trial paper that was published in The Lancet. We were told they’d been instructed to include key words and phrases that were important for the marketing of the drug. It also became apparent that the people who were involved with the companies marketing the product were also key advisors to the World Health Organization (WHO), the European Medicines Agency (EMA), and others. Roche, together with some other manufacturers of influenza drugs, are the funders of a group called the European Scientific Working Group on Influenza (ESWI). This industry-funded group wrote WHO’s first influenza pandemic preparedness plan.
So, as a result of this investigation we did a follow up piece about the conflicts of interest of the advisors.(2) This proved even more controversial than the first, and we came under attack. The WHO would not release any information about whether their advisors declared conflicts of interest or not. The scientists said they had, but WHO would not confirm or deny.
Embarrassingly I ended up on an American “shock jock” show, where they took the line that this is a big pharma conspiracy, which was very far from what we had intended with this story. We’d been very careful to make the point that when you are making policy decisions, which could have huge impact in terms of finances and patient harm, should you really be using the people who have been marketing the drug? You would think not.
But the attacks continued. Nature did an exposé,(3) and people tweeted that our research had been discredited. In our article there had been a sentence that maybe could have been worded more carefully, so we ended up conceding one or two points in relation to the story, and the result was taken as an admission that our story was wrong, starting a back and forth between Nature and the BMJ. It became one of the most read articles on the BMJ website at the time.
The next thing we looked into was medical device regulation. We started with the European Union directives, which took some reading. Ours was a joint investigation between the BMJ and Channel 4’s Dispatches, so we had a whole team to help us go through the directives. Then in 2011 a very useful study was published by a group at the Cleveland Clinic, in Ohio. They looked at the US Food and Drug Administration (FDA)’s regulatory system for medical devices. The FDA have two processes, one is pre-market authorisation, and the other is called 510(k). To get a device onto the market using pre-market authorisation you have to have clinical evidence, such as from randomised control trials, or a case control study. But for 510(k) you only need to show something called equivalence, which means that your new device is “substantially equivalent” to an old one, a predicate device, that is already authorised.
It is possible to trace the history of some 510(k)-authorised devices, and to track the tweaks and changes that have been made to the design over time, so that a device approved in 2015 might not have had any clinical studies to support it, and gone through so many changes since, say, 1980, when the original version was approved, that you can end up with virtually a whole new device but without any clinical evidence for it. The Cleveland team’s study found that two thirds of device recalls were in respect of devices that had gone through the 510(k) route, that is, there was no direct clinical evidence for those versions of the devices.(4)
Working with the Centre of Evidence based Medicine at Oxford University, we thought it would be interesting to replicate this study in Europe. But we couldn’t. Because we kept coming up against the words: “commercial, in confidence”.
Devices come in different risk categories. In Class I, you have low-risk items like plasters and syringes; in Class II there are insulin pumps. Class III is for hip implants, pacemakers, things that need surgery to get them in place. I phoned the Medicines and Healthcare Regulatory Authority (MHRA) and asked if they had a list of Class III devices that are used in the NHS.
And they said “No”. No such list exists.
We knew that some implants must have been causing problems, so we put in a freedom of information request to look at reported harms. Again, “commercial in confidence”. We learned that under the Freedom of Information Act you can get clinical data related to pharmaceutical trials, but you can’t get it for medical devices.
So, we were thwarted from the outset by a lack of transparency. We wrote a story(5) about a hip implant called the ASR, which is made by DePuy, a subsidiary of Johnson & Johnson, and which was recalled in 2010. The ASR was used for two types of orthopaedic surgery – total hip replacement (ASR XL) and hip resurfacing (ASR resurfacing). The ASR in the United States had gone through 510(k) for total hip replacements. For the hip-resurfacing application it should have gone through the full pre-market authorisation process, but the FDA had called a halt to the approval because of problems with the trial. That didn’t happen in Europe. It has been used in tens of thousands of patients.
We later looked at all large diameter metal on metal total hip replacements. We reviewed the medical literature, and whole 510(k) daisy chains, and found that among an entire class of implants there was no publicly available clinical data to support their use. Now, bear in mind that these are made of cobalt and chromium. In the 1970’s and 1980’s surgeons were debating the toxic effects of cobalt and chromium, and in 2007 a comment piece in the Lancet(6) pointed out that little is known about the transport, distribution, excretion of metal ions in the body; toxic effect thresholds have not been characterised. So wouldn’t you think there would be a need to proceed with caution? We know a lot about the local effects of metal ions in the body, but about the systemic effects we know very little because if we’re talking about pharmacovigilance of medical devices like you have with drugs, well, it just doesn’t happen.
My question was, in this article, surely we should proceed cautiously with these devices, given that there is no known toxic threshold for these compounds? In the media it became, the BMJ hip replacement cancer scare, which had not been our intention at all. Suffice to say, that the bullying I encountered as a result gave me a glimpse into the business side of orthopaedic surgery and how things run when there is a great deal of money at stake, and some of it is very unsavoury.
After that, we went under cover. I should point out that it is actually really hard to get clearance to go under cover. There has to be a strong public interest reason, and you have to demonstrate that you can’t get the information any other way. We had a good reason – this was health. We created a fake metal hip. We called it the TMH (total metal hip) and tried to get our metal hip onto the market. We had a fake dossier, and gave it some disastrous data, failures, ions all over the place.
The system here in Europe is that you have to go through what are called notified bodies. There were about 70 of these, they are private organisations, and these are where manufacturers go to get a CE mark. Once they have this mark they can sell the device anywhere across Europe. The various regulators in the different countries then regulate those bodies. It’s an incredibly complex system.
You have to pay these bodies to review your dossier, before they decide if it should receive a CE certificate. That already seems like a conflict of interest, as these companies are paid for approving the device and then are paid again to do follow up audits every few years. An officer at one of the notified bodies in the Czech Republic admitted to us that it is “on the side of manufacturer and their products, not on the side of patients.”
We set up a fictitious company called Changi, and we hawked our new device around notified bodies, pretending to be their PR people. We found that not only are regulators out-sourcing the regulation of medical devices to private companies, but some of these private companies are further out-sourcing their device regulation activities to other private companies. We found ourselves at a branch of a Czech notified body which was in South Korea, where a small group of companies came together to operate as a one-stop-shop. Device manufacturers would go to one for advice about how to create their dossier, then another to receive the CE certificate, and then the next company offered marketing. We were told that it is easy to get products approved in Europe, and they are not worried about inspections by European officials, though they were really scared of the FDA.
We got our design approved for a CE certificate for our non-existent hip implant. That investigation had an impact.(7) We presented our evidence at the Science and Technology Committee and at the European Parliament when they were updating their medical device regulations.
Sugar and water
We were at a meeting just before the 2012 Olympics, when sports medics told us that sports drinks are one of the most controversial thing ever. What could be controversial about sweetened water? Again working with the Centre of Evidence based Medicine, we decided to take the top 10 best selling UK and US sports and fitness magazines, and pulled out the adverts to see what health claims they were making for sports drinks. Then we looked at the references to see what evidence we could find to support them. Then we tried the manufacturers’ websites to find references, before approaching the companies for evidence to support their claims.
We were told things like, water doesn’t quench thirst.
Anecdotally, we heard that one company was telling kids that if they drank water rather than sports drinks they would get cerebral oedema. An idea had been created, that of ‘exercise induced dehydration’. Now, any doctor who has treated someone with genuine dehydration, well, it’s not quite about jumping on the treadmill and getting a bit thirsty. But this idea had been created that if you exercise you get exercise-induced dehydration, and the solution is to drink more sports drinks. This then led to a genuine health concern which is, hyponatremia – a reduction in the body’s electrolyte levels due to drinking too much, which was leading to cerebral oedema, and there was a study in the New England Journal of Medicine that looked at the Boston Marathon and people who had had exercise-induced cerebral oedema, because they’d actually been taking in too much fluid.(8)
The backlash against our “The truth about sports drinks”(9) was incredible. The attacks kept on coming. One company held up what they said were over 100 clinical trials that underpinned their science. We critically appraised all of them, and found that most were poor quality and didn’t support the main claims made.
We did a programme with BBC’s Panorama about in vitro fertilisation (IVF).(10) You get to a certain age and lots of your friends are going through IVF, and if you listen to their stories you find they’re all having different kinds of treatments and also being sold extras, on top of IVF, so-called “add-ons”. We again teamed up with Oxford University, we scraped lots of websites from fertility clinics looking at the claims made for add-on treatments, and then looked at the evidence. We found very few were able to support their claims that these add-ons could actually improve your chances of having a baby. We used undercover again – and came under criticism for this – to look at the lack of fully informed consent for these treatments, and the lack of evidence, and how poorly regulated they are by the HFEA. Needless to say we experienced a backlash from private clinics in the UK.
There are times when you get home, shut the door, and you’re looking at the door, wondering what’s going to happen next. You can take the professional attacks, but worse than that is the sexual harassment. As a female reporter on a heavyweight subject I have my authority questioned, because I’m young and female, and I am constantly challenged about my ability to carry out these investigations. It’s a constant battle, and that is probably the biggest challenge in this area.
Based on Deborah Cohen’s presentation at the HealthWatch 2017 AGM, adapted by Mandy Payne
1. Cohen D. Complications: tracking down the data on oseltamivir. BMJ 2009;339:b5387. http://www.bmj.com/content/339/bmj.b5387
2. Cohen D, Carter P. WHO and the pandemic flu “conspiracies”. BMJ 2010;340:c2912. http://www.bmj.com/content/340/bmj.c2912
3. Butler D. Flu experts rebut conflict claims. Nature 2010;465:672-673. http://www.nature.com/news/2010/100608/full/465672a.html
4. Zuckerman DM1, Brown P, Nissen SE. Medical device recalls and the FDA approval process. Arch Intern Med. 2011 Jun 13;171(11):1006-11. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/227466
5. Cohen D. Out of joint: The story of the ASR. BMJ 2011;342:d2905. http://www.bmj.com/content/342/bmj.d2905.full.print
6. Learmouth ID, Case CP. Metallic debris from orthopaedic implants. The Lancet 2007;369(9561):542–544. http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(07)60251-7/fulltext
7. Cohen D. How a fake hip showed up failings in European device regulation. BMJ 2012;345:e7090. http://www.bmj.com/content/345/bmj.e7090
8. Almond CSD et al. Hyponatremia among Runners in the Boston Marathon. N Engl J Med 2005; 352:1550-1556.
9. Cohen D. The truth about sports drinks. BMJ 2012;345:e4737. http://www.bmj.com/content/345/bmj.e4737
10. BBC One Panorama. Inside Britain's Fertility Business. First broadcast Monday 28 November, 20:30. http://www.bbc.co.uk/programmes/b084ngkd (not currently available).
Why is it controversial to tell the truth about health care?
Peter Gøtzsche, physician, medical researcher and leader of the Nordic Cochrane Centre at Rigshospitalet in Copenhagen, Denmark, received the 2016 HealthWatch Award at the charity's 28th Annual General Meeting on Thursday 20th October 2016 at the Medical Society of London.
This article is adapted from a transcript of his talk.
People ask me, why do you look for controversies? And I tell them, I don’t, they come to me. My work is something like that of a medical detective. People come to me if they feel something is wrong in healthcare. When I start looking into these issues, I usually dig very deep. I find skeletons, and when I expose these skeletons, the people who buried them can get very angry.
Even in my most routine work, I dig deep and I still find skeletons.
I started digging many years ago when I looked into drug trials of the NSAIDs – arthritis painkillers. These were head-to-head trials, comparing one drug with another. I collected every trial I could find, and used meta-analytic methods to find out if the results were plausible. I found that they were not, in a statistical sense, and I published my conclusions in my thesis. Doug Altman, professor of statistics in medicine at the University of Oxford, saw my work and in 1990 he sent it to his colleague Iain Chalmers. I’d published three meta-analysis by this time and felt I was quite prolific, but Iain told me his group had published hundreds in Cochrane, a database I’d never heard of. He invited me to Oxford for the 1992 opening of the UK Cochrane Centre, of which he was the director. In 1993 we opened the Nordic Cochrane Centre and four years later I became a full time researcher there, reasoning that as a doctor I could only help one patient at a time, but if I did a good review I could help one hundred thousand.
Shocking results were not unusual
It was felt that my results with NSAIDS had been so shocking that it must be something that only applies to arthritis drugs. But this was the first time anyone had trawled through a whole medical area and done a statistical overview, and we would soon find out that the phenomenon was not unusual at all. In 1997 my wife Helle, a clinical microbiologist, did a study on antifungal agents for people with, for example fever, cancer, or neutropenia, who were at risk of dying from a fungal infection. We did a routine comparison between fluconazole and Amphotericin B, the two most promising drugs at that time. We found that some of the trials were three-armed, with a third arm consisting of nystatin, which is effectively no more than a placebo. The manufacturer, Pfizer, had lumped the data from a drug that didn’t work, with amphotericin B to show that their drug, fluconazole, was better than when you handicapped your comparator. What is more, they had also given amphotericin B by mouth, even though it is poorly absorbed and not prescribed orally with these patients. We tried to talk to the investigators about our results, and they referred us to Pfizer, who wouldn’t answer our questions.1
We couldn’t even see whether one trial publication was re-using the same patients that were included in another trial publication, so we might be counting the patients twice. We asked Pfizer about that, too, and they didn’t reply. We published it in JAMA2 and Helle presented it at a conference on microbiology in Toronto in Canada. There were 800 people and her presentation went way over time, not because she over-ran, but because there were so many questions. Drummond Rennie, one of the JAMA editors, wrote in an editorial that “fluconazole raced against a heavily handicapped opponent”.3 In a response letter to JAMA, Pfizer conceded that the comparison drugs did not work and that it would change how it reported results of trials.4
So, we ended up in the New York Times because we’d reported misleading research being published by the biggest drug company in the world.5 Two years later we were there again.6 My PhD student had studied the placebo effect. We’d thought it could be powerful, as did most doctors. But most doctors make the mistake of asking, what is the placebo effect? And then attempt to answer it by observing what happens to a group of patients treated with placebo. That is wrong. Think about regression to the mean, and the way many conditions heal by themselves.
We couldn't find evidence for placebo effect
Well, that is NOT placebo. To study placebo, you need to take an untreated control group, and compare that with the placebo group, then in most trials you can also have a third group, using an active drug. We collected results from 113 trials, we published in the New England Journal of Medicine and the Cochrane Library. We couldn’t see much effect with placebo. Maybe a little effect on pain, but we didn’t even know if it was a true effect because you cannot blind an untreated control group, and when you measure subjective effects people tend to hope that they’ll get some benefits because they think they are receiving a drug. So, we couldn’t find evidence for placebo effect. It was terrible for the placebo community and made a lot of work for us, dealing with questions and criticisms.
There is too much unreliable research published by people in order to sustain their erroneous beliefs. So it is fundamental that we try to persuade politicians that we cannot leave people to be their own judges. I can’t go to the car inspection with my 18-year-old car and reams of paper and say, I’ve already done the inspections myself, no need for you to do it! Yet that is how we approve drugs. I have worked a lot with psychiatric drugs, and I published a book last year on the subject,7 and there will be documentaries to follow. People are starting to realize that psychiatric drugs are not the solution to psychiatric problems, in fact they make them worse. They disable the brain. People may think it’s useful that they do this because the effect is to sedate the patient, but they also cause many side effects. The clinical trials in psychiatry are the worst I have ever seen. And the more psychiatric drugs we use, the more people end up on disability pensions because they can’t work. There is more and more pressure to establish helplines for people who want to come off psychiatric drugs, so things are moving in the right direction.
On mammography screening: I think it should be stopped. In 1999 a Swedish study compared regions which had taken up mammography screening early, and they compared them with other regions that hadn’t implemented it yet. They couldn’t find any difference in mortality.8 At that time we had screening in 20% of Denmark. The Danish Medical Association asked the Danish Board of Health, can’t we trust the randomized trials which said the screening worked? That hot potato landed in my lap. We were asked to review the randomized trials of mammography screening. Five weeks after I had this assignment, there would be questions in Parliament about whether we should have screening.
A real "Yes, Minister" moment
We worked fast – in four weeks we had produced a report, with meta-analysis, and it concluded that we cannot exclude the possibility that mammographic screening does more harm than good. We took it to the National Board of Health and were immediately called to a meeting, with the result that two days later we were told that our paper was a non-paper. It didn’t exist. This was a real “Yes, Minister” moment. Fortunately, we were able to prove that it really did exist, because we’d sent it to the Danish Board of Health by messenger and its delivery had been recorded! I was told to change my letter, to say there had been misunderstandings, and to say that the report was only preliminary. I didn’t accept that. A promise was made to send it to the Minister of Health. It never arrived.
Two weeks later the scandal broke loose. Journalists got hold of both copies of the letters, my original and one that had been changed and which I had not signed. Both on a full page of the newspaper. As you all know, “Yes Minister” tells you how government discredits an unwelcome report. Step one – refuse to publish in the public interests, saying you are waiting for the results of a more detailed report, still in preparation. If there isn’t one, commission it! And that’s what happened. We were told to prepare a Cochrane review about this, and our original report would not see the light of day. So, we did a little more work, we published our findings in the Lancet,9 then two years later we published the Cochrane review which, naturally, confirmed the original findings. But we were blocked.10 [Editor’s note: the paper was removed completely from the journal website without any formal retraction. The authors later published it in the Danish Medical Bulletin11] You can see why, when life gets tough, I like to watch “Yes Minister”.
I published another paper in the Journal of the Royal Society of Medicine last year, calling for mammography screening to be stopped.12 What does breast cancer screening mean for total mortality? Mortality rates with and without mammography screening are identical. Except, of course, if you include deaths caused by screening overdiagnosis – if you do that, you find that screening increases mortality. Why deaths by overdiagnosis? Well, if you are diagnosed with breast cancer you get chemotherapy and radiotherapy, and that kills some women. Do you think you could have sold mammography screening if this was what you’d shown to the politicians 25 or 30 years ago? And because the NHS leaflet on breast screening was no good, we produced a new leaflet and published it in the BMJ, it’s been translated into many languages and it’s on our website.13
Screening is the best way to earn money, if you are a private doctor
But of course screening – to investigate and find illnesses before people even know they are ill – is the best way to earn money, if you are a private doctor.
So I thought we should also look into health checks, looking at morbidity and mortality from disease. To our big surprise there were already 16 randomised trials with many participants and long follow-ups – 9 years – and almost 12,000 deaths, so much material! And what did we find? No effect whatsoever. Total mortality, around 1, and the same for cancer, cardiovascular mortality. We published in the BMJ.14 Then a huge Danish study was published after our review, that one included 3,000 deaths, and it didn’t find anything either. Extensive screening for risk of ischaemic heart disease followed by repeated lifestyle counselling, a beautiful ten-year study that was also published in the BMJ.15 So there are no lives saved by screening but there is a lot of harm because you make diagnoses and then treat them with drugs that might make your patients impotent, etc., when they might never even have noticed the symptoms of the original condition or just put it down to old age.
Look at England – there is a universally applied health check for everyone aged 40-74. It was launched in 2009. The Department of Health argued that by spotting people who are at risk from heart attack, diabetes, stroke, kidney disease, these conditions could be prevented. Finally we’d had enough and wrote to the Times, who published on the front page to say that health checks are utterly useless.16 They now claim that, although the programme is not supported by direct evidence from randomized controlled trials, there is an urgent need to handle the growing burden of disease associated with lifestyle behaviours. Six months later, NICE was helping local authorities to encourage people to attend health checks and to support them making changes to improve their health.
Now, this is how the mob operates. You don’t ask the boss questions. You just do what you are told, otherwise you might get a bullet in your head. NICE has prostituted itself to the politicians’ message. This is a side of medicine that students aren’t taught about.
I am kind of fed up with being introduced all over the world as controversial. Why is it controversial to tell what you find when you try to do good science?
Peter Gøtzsche’s books include Deadly Psychiatry and Organised Denial (People's Press, 2015); Deadly Medicines and Organised Crime: How Big Pharma has Corrupted Healthcare (Radcliffe Publishing, 2013); and Mammography Screening: Truth, Lies and Controversy (Radcliffe Publishing, 2012).
The HealthWatch Award is presented annually to an individual who has made significant steps either in medical research or in improving the public’s understanding of health issues by clarifying complicated and often misunderstood medical matters for the general public.
1. Gøtzsche P. Deadly Medicines and Organised Crime. Radcliffe Publishing, London, 2013
2. Krogh Johansen H, Gøtzsche PC. Problems in the Design and Reporting of Trials of Antifungal Agents Encountered During Meta-analysis. JAMA. 1999;282(18):1752-1759 http://jamanetwork.com/journals/jama/fullarticle/192088
3. Rennie D. Fair Conduct and Fair Reporting of Clinical Trials. JAMA. 1999;282(18):1766-1768 http://jamanetwork.com/journals/jama/fullarticle/192075
4. Panzer H. Improving the Conduct and Reporting of Clinical Trials. JAMA. 2000;283(21):2787-2790 http://jamanetwork.com/journals/jama/fullarticle/1030783
5. Grady D. Medical Journal Cites Misleading Drug Research. New York Times, 10 Nov 1999 http://www.nytimes.com/1999/11/10/us/medical-journal-cites-misleading-drug-research.html
6. Kolata G. Placebo Effect Is More Myth Than Science, Study Says. New York Times, 24 May 2001 http://www.nytimes.com/2001/05/24/us/placebo-effect-is-more-myth-than-science-study-says.html
7. Gøtzsche P. Deadly Psychiatry and Organised Denial. People's Press, London, 2015
8. Sjönell G, Ståhle L. Hålsokontroller med mammografi minskar inte dödlighet i bröstcancer. Läkartidningen 1999; 96: 904–13.
9. Gøtzsche PG, Olsen O. (January 2000). "Is screening for breast cancer with mammography justifiable?". Lancet 2000;355:129-134 http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(99)06065-1/abstract
10. Zahl PH et al. WITHDRAWN: Results of the Two-County trial of mammography screening are not compatible with contemporaneous official Swedish breast cancer statistics. European Journal of Cancer 2006 Mar 9 [epub ahead of print] https://www.ncbi.nlm.nih.gov/pubmed/16530407
11. Zahl PH et al. Results of the Two-County trial of mammography screening are not compatible with contemporaneous official Swedish breast cancer statistics. Danish Medical Bulletin 2006;53:438–440 https://web.archive.org/web/20070621235925/http:/www.danmedbul.dk/Dmb_2006/0406/0406-artikler/DMB3890.htm
12. Gotzsche PG. Mammography screening is harmful and should be abandoned. J R Soc Med 2015 Sep;108(9):341-5. http://journals.sagepub.com/doi/abs/10.1177/0141076815602452
13. Nordic Cochrane. Mammography screening leaflet, 2012. http://nordic.cochrane.org/mammography-screening-leaflet
14. Krogsbøll LT et al. General health checks in adults for reducing morbidity and mortality from disease: Cochrane systematic review and meta-analysis. BMJ 2012;345:e7191. http://www.bmj.com/content/345/bmj.e7191.long
15. Jørgensen T. Effect of screening and lifestyle counselling on incidence of ischaemic heart disease in general population: Inter99 randomised trial. BMJ 2014;348:g3617 http://www.bmj.com/content/348/bmj.g3617
16. Smyth C. NHS checks on over-40s condemned as ‘useless’. The Times, 20 August 2013 http://www.thetimes.co.uk/tto/health/news/article3847530.ece
Why evidence matters
The 23rd HealthWatch Award was presented to Dr Mark Porter MBE; GP, journalist, media doctor and presenter of BBC Radio 4’s ‘Inside Health’.
Receiving his award at the Medical Society of London, Dr Porter gave an insight into his unique career, and explained that his listeners, readers, and patients are why evidence matters.
‘For me, it’s not about the data, but how it influences clinical practice — what difference is made to the man or woman sitting in front of me in my consulting room.
Like most, I was a naïve young doctor who did what his boss told him, whose boss did what his boss told him because it was what his boss did, and that’s how we based a lot of our practice. It’s been a gradual transition to the importance of evidence, and now, I’m a complete convert.
My job is not to be an expert. It is to put experts into the public domain. It is to ask questions, and to give them the chance to explain the rationale behind evidence-based medicine; what it is and why it matters to the public, so that everybody out there can make an informed decision about their future.
I don’t need to sell evidence to my listeners — they absolutely ‘get it’. Less straightforward is the way that research is reported. It’s hard to see through the veneer of gloss that’s put upon it by the journalists, or the journal, or by the researchers themselves. Press releases from journals ‘big things up’ — it’s not surprising that the story gets slightly confused.
Significance is not well understood. I’m not talking about statistical significance here; I’m talking about what difference this finding might make to their lives. If something doubles the chance of you getting something, but your chance of getting something is pretty near zero in the first place, it’s irrelevant.
Terminology is a struggle. I’m hopeful that there will be a third series of ‘Inside Language’, a series with Professor Carl Heneghan and Dr Margaret McCartney looking at terminology, from surrogate markers to t-tests, which has proved popular.
My job is to teach the public to be a little bit sceptical, a little bit cynical about what they read, look for vested interests, understand why something might be, and understand how to ascertain if it’s relevant to them.
But, I do have concerns, and I just want to share a couple of those with you.
The first is the relevance of the data.
As we bow to the altar of evidence-based medicine, sometimes we’re blinded by the light. We need to be critical about the relevance of the data that we’re looking at, in terms of clinicians like me, working at the coalface of the NHS. A lot of that data ends up informing and producing guidelines, and we need to look at how rigidly we adhere to those guidelines when faced with very different individuals.
The vast majority of patients that I see, perhaps 90%, wouldn’t get into a clinical trial because they don’t meet the criteria - yet they’re the patients that we’re treating. Still a large proportion of the data that we’re looking at is coming from on high. We don’t know how relevant it is to the people we’re actually seeing on a day-to-day basis.
An example I use is that of the contraceptive pill. The early explanatory trials show a failure rate of 1 in 300, so if you had 300 women on the pill for a year, one would expect to get pregnant. But in the real world, where trial conditions don’t apply and all sorts of things arise, that failure rate approaches 1 in 10. That’s a huge, huge difference. But it’s the sort of difference that patients can grasp straight away, and suddenly they see the rationale behind long acting contraceptives.
I’d like to see more pragmatic trials — but they’re not the be all and end all, either. I just want to emphasise that the evidence that we’re looking at is not always pertinent to the people that we’re treating.
My second concern is that the selective data is then used to produce the guidelines. I’m all for guidelines. I’m all for best evidence based practice. But I worry about how rigidly we apply it, because of external pressures. We have QOF, prescribing initiatives, pressure to follow published guidance. Even if the guidelines were perfect - how do we know that we’re applying them correctly to the person that’s sitting in front of us?
External pressures mean that guidelines get rigidly applied. They become tramlines. And I think that’s a problem.
I recently met an elderly gentleman with a humeral fracture in a falls clinic. He had diabetes, atrial fibrillation and other conditions, and had felt lightheaded for months. A number of doctors, consultants, had seen him and treated him according to the guidelines. He was taking sixteen different medicines, including six that lowered his blood pressure. I have no doubt that getting blood pressure to target improves outcomes for 1,000, 10,000, 100,000 diabetic patients. But that doesn’t tell me what’s going to happen to the patient sitting in front of me. He could have hit his head, or broken his hip. We could have killed him. That’s something that we need to consider in general practice.
Scepticism doesn’t stop once you have some evidence, or have guidelines.
I don’t have answers. If pushed, I’d like to revert to that basic tenet of clinical practice: Primum non nocere — or, as I say to my patients — if in doubt, do nowt.
My topic is my libel trial and some of the battles I’ve had with alternative medicine, alongside my colleague and friend Edzard Ernst.
To start with, I should stress that my background is not in medicine, but in particle physics. And, for most of my career in journalism, I focussed on promoting good science – exciting, interesting, challenging science – the kind of science we used to dream about when we were kids.
For example, a few years ago I heard a song on the radio by Katie Melua, called nine million bicycles, which contained the following verse.
We are 12 billion light years from the edge,
That’s a guess,
No one can ever say it’s true,
But I know that I will always be with you.
It made me a little bit annoyed, because the song implies the universe is 12 billion years old, and it’s not. Then she says, “That’s a guess,” but it’s not a guess - this is science and scientists don’t make guesses, we make careful measurements. Next she says, “no-one can ever say it’s true”. But that’s not right because in science we always try to get closer to the truth. And finally, “I know that I will always be with you”. Well at this point I felt I couldn’t trust a word this woman says. So I wrote an article about this for the Guardian, and at the end I re-wrote the lyrics to make them a bit more accurate.
We are 13.7 billion light years from the edge of the observable universe,
That’s a good estimate with well defined error bars,
Scientists say it’s true, but acknowledge that it may be refined,
And with the available information, I predict that I will always be with you.
The great thing was that Katie Melua actually read the article, she got the joke, and we met up and she re-recorded the song, using my lyrics. Even better, the article about my re-write hit a big mainstream audience, so I was able to convey a bit of cosmology to the general public.
However, about ten years ago, instead of finding ways to communicate mainstream science, I began writing about pseudoscience. I started to get interested in some different kinds of claims that were being made in the name of science, that weren’t very scientific, but which could have serious consequences. For example, I wrote about homeopaths who offered homeopathy pills as protection from malaria. Moreover, I worked on an undercover investigation with Sense About Science and we exposed the dangers of the advice that we found some homeopaths were giving to people travelling to malaria-affected countries.
Back in those days I still didn’t know that much about alternative medicine generally, so I was interested to see a BBC 2 show in 2006 called “Alternative Medicine – The evidence”. It was particularly memorable, because the narrator explained an astonishing clip: “In China this woman is having open heart surgery. But it is unlike anything you will see in the West - she’s still conscious. Because, instead of a general anaesthetic, this 21st Century surgical team are using a 2000-year-old method of controlling pain. Acupuncture.”
I thought, either that’s the most staggering bit of film I’ve seen all year, or there’s something wrong here. I did a bit of digging and I learned that the Royal College of Anaesthetists had written a report for the BBC about that very piece of footage, and it said that the patient didn’t have a general anaesthetic, but she had in fact received sufficiently large doses of sedative and painkilling drugs to make the acupuncture’s role no more than cosmetic.
It was around this time I met Edzard Ernst, Professor of Complementary Medicine at Exeter University. We soon realised that we had overlapping interests, namely an appreciation of the importance of evidence and the desire to communicate research to the general public. In time, we wrote a book together1 to look at the evidence for alternative medicine, to consider what does and doesn’t work, and whether any of it could actually be dangerous. We had chapters about homeopathy, acupuncture and all kinds of other practices, and also we wrote articles in the press, to help people weigh up the risks and benefits of these treatments.
One of the articles I wrote at the time was on the subject of chiropractic and was published in the Guardian on 19 April 2008, with the title "Beware the Spinal Trap". I mentioned the way that many chiropractors go from claiming to treat back problems to being able to treat things like ear infections and asthma, based on this rather odd philosophy that manipulating the spine is able to influence the rest of the body via the nervous system. As a result of writing this article I became the subject of a libel action brought by the British Chiropractic Association (BCA).
The case went on for two years, and I was facing costs of £500,000 were I to lose. Fortunately, the BCA finally backed down, withdrew its lawsuit, and the case ended. It was a miserable time, causing a huge amount of stress and forcing me to dedicate half of my working time on the legal battle. Nevertheless, some good things came out of this. Firstly, it encouraged discussion and the challenge of chiropractic claims. Fiona Godlee, in the British Medical Journal, published a debate between Edzard Ernst and the BCA. She concluded: “Readers can decide for themselves whether or not they are convinced. Edzard Ernst is not. His demolition of the 18 references is, to my mind, complete.”2
Suddenly doctors were becoming more aware of what chiropractors were claiming. There were also campaigns run by people who are not really part of the medical establishment. For example, Simon Perry, Chris French, and Alan and Maria Henness of the Nightingale Collaboration, submitted 600 legitimate complaints about chiropractors to the General Chiropractic Council. Almost overnight, chiropractors reacted and began taking down their websites and withdrawing leaflets in order to avoid prosecution for claims that could not be supported by evidence.
It was around this time that the campaign to reform the English libel law started. There was a general sense that it was too easy to bring a libel case, and that this state of affairs was actually discouraging critical media reporting on matters of serious public interest. The burden of proof was on the writer, there was no public interest defence, and it was possible for companies to sue individuals and land them with high costs, with no risk to the organisation bringing the case.
What is more, a lot of these features were peculiar to English libel law, so if someone anywhere in the world wanted to issue a writ, they would be tempted to issue it here in London. This is what is known as libel tourism. Even the United Nations had condemned the English judges' practice of welcoming rich libel tourists from across the world to their hospitable courts; and following a case in which an English judge ordered the censorship of a New York author's book on terrorism, which had not even been published in Britain, the US Congress began drafting a law which would guarantee that English libel judgments have no validity in America.
My case was not unique. At the time, some of the other people being sued for libel or being threatened included Ben Goldacre, Peter Wilmshurst, David Colquhoun, Andy Lewis, Mumsnet. Even the prestigious journal Nature was sued over an article on cosmology, and that case also lasted for two years and ran up huge costs. What is more, there were many good articles which were never even written because of self-censorship fuelled by a fear of a libel suit – what we call the “chilling effect”.
The Libel Reform Campaign was founded by Sense About Science, English PEN and Index on Censorship, with support from groups ranging from HealthWatch to Skeptics in the Pub, from academic journals to Mumsnet, and there were people all around the world who joined in, because they realised they could also get dragged into London’s Royal Courts of Justice. The Libel Reform campaign at www.libelreform.org received the backing of the editors of BMJ, Nature and New Scientist, and people like Richard Dawkins, Ricky Gervais, Stephen Fry, Penn & Teller, James Randi, Sir Tim Hunt, Sir David King, the UCL Provost and 50,000 others.
After a remarkable grass roots campaign, the Defamation Act received royal assent on 25 April 2013, and became law in England and Wales at start of 2014. This change to the law of defamation aims to strike a fair balance between the right to freedom of expression and the right to reputation. Plaintiffs now have to show that they suffer serious harm before a court will accept the case. There’s protection for website operators, a defence of 'responsible publication on matters of public interest' and new statutory defences of truth and honest opinion.
But there is still work to be done. In Northern Ireland there remains a problem because Belfast’s libel laws have not yet been brought into line with the new Act. So the campaign is still ongoing, and with the backing of HealthWatch and all our other supporters, I am optimistic that Northern Ireland will realise that necessity of bringing its defamation laws into the twenty first century.
1 Singh S and Ernst E. Trick or treatment? Alternative medicine on trial. Corgi Books, London, 2008.
2 Godlee F. Keep libel laws out of science. BMJ 2009;339:b2783. See: http://www.bmj.com/content/339/bmj.b2783.full
Sir Richard Doll receives HealthWatch award
The Fourth Annual HealthWatch Award was presented to Professor Sir Richard Doll OBE MD DSc FRS of the University of Oxford, at the HealthWatch Annual General Meeting on 14th October, in recognition of his 50 years of work at the highest level in clinical epidemiology.
It was Sir Richard who, in the 1950s, led the landmark research study of 34,000 male doctors which showed that people who smoked were significantly more likely to develop lung cancer. He has more recently studied links between vitamin D and the brittle bone condition osteoporosis.
The presentation of the HealthWatch award followed Sir Richard's fascinating talk, "Help and hindrance in epidemiology". The help, he said, came from GP's, who are highly cooperative when invited to give information for epidemiological studies. Patients are similarly willing - out of thousands of patients studied he recalled only one letter which asked how the scientists had obtained their information.
Employers, on the other hand, have a tendency to "lose" records of past employees who may have been exposed to dangerous materials such as asbestos or poison gases. He found that questions asked in the House often prompted renewed searching and the subsequent discovery of missing documents.
Fears about patient confidentiality can also hinder epidemiological research. A study which hoped to investigate the possibility of a link between road traffic accidents and tranquilising medication ran into difficulties when the police questioned the ethics of looking at details of patients' medical prescriptions.
The Department of Health have raised objections to anonymous HIV testing of blood samples routinely taken on hospital admission; and a study of the effect of natural radon emission on lung cancer was interrupted by a non-research-minded committee who insisted that they alone should be responsible for choosing the subjects for the trial.
Sir Richard's talk was beautifully delivered and provided an enjoyable and thought-provoking conclusion to this year's AGM.