For outstanding leadership in the pursuit of medical truth

What’s wrong with the medical literature and what can we do about it?

This is an enormous honour. HealthWatch is a charity of which I’ve been fully aware for a very long time. As Peter Wilmshurst said to me earlier, now I’ve joined the “awkward squad”!

Many of my heroes are big parts of HealthWatch. Everything I say tonight is absolutely on the shoulders of giants, many of whom are in this room, and who have been unafraid to upset people and to challenge vested interests in the cause of health and scientific endeavor. HealthWatch shares many noble aims with the BMJ. We had a strategy session at the BMJ just yesterday and amongst our ground rules the very first was that all statements will be based on evidence, and another one that came up was, there shall be no sacred cows. I genuinely think that we should have no sacred cows.

The BMJ has for a long time been a campaigning journal, and that is something we are continuing. I’d like to talk about two campaigns in particular tonight, one is, open access to clinical trial data, and the other is “too much medicine”, which again fits very well with HealthWatch’s aims. This is one of many areas in which we find that the evidence base is distorted, and it ends up pushing us to over treat and over diagnose - this conspiracy of enthusiasm for treatment which I think as medics we have to constantly rein ourselves back on. And the other thing I’m going to speak about tonight is the theme of patient partnership - trying to bring patients into everything we do.

With these campaigns we feel we can bring to bear quite a unique mix of science and journalism, and we have the original research, we have our commentaries, we have investigative journalism which has allowed us to dig that bit more and hopefully to pull down some of these sacred caws that otherwise might go unchallenged.

So to my theme. We have a problem. We want to practice evidence based medicine, but the evidence on which our decisions are based is flawed. It is incomplete and it is of poor quality and based on hidden data. There is a huge loss of trust. There have been too many examples of bad practice, bad faith and out-and-out misconduct, especially but by no means only on the part of industry, over the past 30 years or so. Journals must accept a good chunk of the blame, as must the medical profession, and the research establishment. So we’ve got growing evidence of the problem of misleading, misreported, incomplete data. We’ve got hidden trial data, and the tendency that this distortion has to lead us to over diagnose and over treat, which is how these two campaigns find themselves coming together.

There was a review in 2010 in the journal Trials,¹ done by Germany’s IQWiG group (the German equivalent of the UK’s NICE), and they looked at a huge number of conditions, and found underreporting and misreporting of trial data across the entire array of medicine, and there was no method that was spared. The problem was very much associated with pharmaceutical company trials, but also non-pharmaceutical company trials, and overwhelmingly they found evidence of benefits being over-stated and harms being understated. In addition we now know that only about half of clinical trials end up getting published² and the US legislation, the 2007 FDA Amendments Act that everyone felt was such a great thing, and is a great thing, is being widely ignored, in the sense that the summary data that is supposed to be published within a year of the study finishing, is not being published within that time frame.³ So we’ve got things in place which should be helping, but they’re not.

I’ll give you two examples, both of which have become “poster children” of this problem.

The ’flu pandemic of 2008-9 gave Roche’s drug Tamiflu an enormous boost. Initially a Cochrane review found it was effective in reducing the complications of influenza, such as pneumonia.⁴ The Cochrane team were asked to update their review, they thought it would be a simple job, just looking at the most recent trials and most likely coming to the same conclusion, but a Japanese pediatrician had noticed that actually the data⁵ on which their review was based was entirely industry-funded, that out of 10 trials that had been summarized in an industry-funded systematic review, only two of these had been published in peer-reviewed journals, in JAMA and the Lancet; the other eight were apparently either unpublished or published only in abstract form. Now these Cochrane reviewers, being the people they are, asked the industry funded reviewer if they could see the data, and were told, “You’ll have to go to the authors of the original trials.” Those authors in turn sent them to the drug company, who said “We can’t give you this.” At which point they came to the BMJ and Channel 4, and we had some articles about this.6 Interestingly, while the European Medicines Agency has approved the label claim that Tamiflu reduces the complications of influenza, Roche’s US website says, with a footnote pointing out that the advice is for American audiences only, that it is not proven to reduce complications.⁶ So you’ve got the same evidence base but the authorizing bodies have come to completely different conclusions.

But what is deeply shocking is that this drug, on which governments are spending billions of pounds, is apparently not yet proved to be any better than paracetamol and may have adverse effects. Moreover, the evidence base on which these decisions are being made is entirely in the hands of the manufacturers of the drug.

Thanks to the intervention of Ben Goldacre’s book⁷ and the huge interest that caused, and building on the work that Iain Chalmers has been undertaking, we are making progress. The AllTrials campaign,⁸ the BMJ’s “open correspondence” page,⁹ which has been publishing all the letters between the Cochrane Collaboration and with Roche, and likewise similar activities of the Cochrane Collaboration with GSK, WHO, and so on, are slowly making an impact. GSK have distanced themselves from the pack and have made some very important concessions.¹⁰ Now Roche, four years after their original promise to make their data available, have done so.¹¹ We hope that we will soon know the truth about Tamiflu. It may be marvelous, we will have to wait to see – the Cochrane Collaboration are planning to publish their revised review in a few month’s time.

The next case is the antidepressant Reboxetine, which used to be quite widely used. Germany’s IQWiG was going to re-license it, in order to keep it on the list of drugs that are reimbursed by health insurance companies, and they asked the drug company Pfizer for the data, and the drug company said, “No.” So they said, if you don’t, we can’t keep your drug on the list. It turned out, when they finally gave up the data, that about 65% of it had never been published. When they took the unpublished data and considered all the data together they found that a drug which was previously considered effective and safe, actually had no benefit.¹²

The difficulty we have is that we just don’t know how much of the current evidence base is similarly flawed.

But it is not just pharma that is at fault. If you have a melanoma you may end up having your lymph nodes dissected out. The procedure, a sentinel node biopsy, is invasive and can result in complications. A study in the New England Journal of Medicine published five year results of the Multicenter Selective Lymphadenectomy Trial (MSLT-I) and these data suggested any survival advantage was marginal.¹³ They were supposed to publish 7-year data, then 10-year data, but the dates for those publications has passed and the authors are not coming up with the goods. We published an article saying, this is a problem, people are having this procedure and the data are not available.¹⁴

So it’s not just industry who are doing this. It is, however, true to say that industry is the largest funder of pharmaceutical trials, and the funder with what we can only call an irreducible conflict of interest. They are there to make money for the stakeholder and, ideally, also to help patients, but that is their conflict of interest which they will never escape.

I think it’s quite important to say that things are changing. Ideally we hope to get patient groups saying, “We’re not going to support treatments without the data.”

There is a long list of initiatives, but the trouble is, many of them are partial solutions, and there’s a sense in which industry in particular has been saying, it’s all solved. And when you look closely at the problem, it’s not solved. One drug company, AbbVie, which produces a drug called Humira for arthritis, is suing the European Medicines Agency to stop them making the summary data around their drug available, and the European Federation of Pharmaceutical Industries and Associations, of which AbbVie is a member, is supporting AbbVie in this.¹⁵ AbbVie’s lawyer has made clear that the company considers even the data on adverse events to be commercially confidential. So, words on one hand, and inaction on the other. There’s no doubt that patient confidentiality is one issue, there are many others, but we can get through them. In fact a lot of the time, if you look at individual examples of information that people are saying they can’t share, it’s actually already being shared, so it’s not as much of a problem as it’s being made out to be.

I expect many people in this room are on statins. The current guidance is to prescribe statins if your 10-year risk of cardiovascular disease is more than 20%. It scoops up a lot of people on these drugs, at enormous cost to the health service and with an enormous revenue for the industry. If that’s what the evidence says, that’s fine. Last year the Cholesterol Treatment Trialists (CTT) Collaboration published in the Lancet an enormous meta-analysis of individual patient data, which concluded that people at even lower risk could benefit from statins,¹⁶ and that led to the suggestion that maybe everyone over the age of 50 should be on these drugs. Now a paper in this week’s BMJ¹⁷ questions that. A new analysis of the data doesn’t find an overall mortality benefit, it also says that none of the trials in the meta-analysis really looked at harms – and statins can cause diabetes, myopathy, and all sorts of minor effects along with some serious harms. They also point out that all the trials in the Lancet meta-analysis are funded by the producer of the statins. Usefully the BMJ paper also lists what low risk patients need to know, and top of the list is the need for lifestyle change. This, in a society dominated by pharmaceutical-funded research, is something that tends to be underplayed and obscured.

Which leads me on to “too much medicine”. A long-standing interest of the BMJ is overdiagnosis and overtreatment – why does it happen? The reasons are very complex: more and more sensitive diagnostic technology, finding that little thing that would otherwise go unnoticed; increased patient expectations; the belief that more medicine is better medicine. And then some less worthy reasons: personal financial gain; doctors paid for doing more, so-called “fee for service”; commercial gain by drug manufacturers and medical device companies; a change in diagnostic criteria so more and more people are labeled as being at risk of or actually having a disease; and conflicted guideline panels, so that those diagnostic criteria are being decided on by people with conflicts of interest.

So it’s no longer just diabetes and hypertension and dementia but we’ve now got pre-diabetes, pre-hypertension, pre-dementia – for which good evidence is lacking but increasing proportions of the population are encouraged to have monitoring and preventive treatment, all of which has side effects, and all of which has costs.

The poster child for overdiagnosis is perhaps breast cancer screening. We’re just beginning to understand why this is a real risk for women, and Michael Baum has been a great advocate for a much clearer understanding of those risks. Thyroid cancer is an example that has received less publicity. It’s one of those cases where you detect something and, of course, it’s very hard to leave something there once you’ve found it, but after surgical removal come all the other problems of treatment - thyroid monitoring , radiation to the neck, so it’s not just one procedure, it’s a lifetime of taking drugs, and being a patient in a way that you wouldn’t otherwise have been.

The BMJ has had overdiagnosis in its sights for some years, and a few years ago we commissioned a news piece to go out on April 1st – a small touch whose meaning some of our readers not in the UK might have missed – it was about the discovery of a new disease called motivational deficiency disorder and it was said to affect one in five Australians, characterized by a severe loss of motivation, so much so, that some of them lost the motivation to breathe.¹⁸ And the great news was that a new drug had been developed, by a Professor Lethargos. One of the recipients of this drug had been so confined by this condition that he hadn’t got off his sofa for two years, but thanks to the treatment is now an investment banker in Sydney. I think most people understood it was a spoof piece. Except for the editor of one New Zealand newspaper who was taken in, and wrote the story up, and when he found out he’d been spoofed he was very upset and wrote me a very rude e-mail – he said , “Credibility is hard-won, Dr Godlee, and you have damaged yours and mine.” Fortunately most of our readers got the joke, and we got some marvelous rapid responses, including one from someone who said, “We discovered this disease two years ago but couldn’t be bothered to write it up.”

Amongst “the thousand natural shocks that flesh is heir to” (a quote from Hamlet) we can daily add new ones, and you can be sure that there is money being made on the way. Cosmetic medicine, having made as much money as possible from women’s faces and breasts, is heading south in more ways than one. We published an article in the BMJ critical of the growing practice of cosmetic genitoplasty¹⁹ by which women have surgery to their labia to make them conform to these new expectations of perfection – so called “designer vaginas”. The day after it was published I received an e-mail.

“What is your problem with women Ms Godlee? This article’s hysterical claims that a lack of testing and rigor in these procedures could result in permanent genital damage are nothing more than misogynist propaganda. Tell you what, the women of the world can keep their hands off your genitals if you keep your hands off theirs. Best wishes, Laurence Shandy, Feminist. I replied:

“Dear Laurence Shandy, Thank you for this. I have no strong opinions on the matter, except that I hope to get through life without surgery to my genitals and think it appropriate for a medical journal to point out the potential dangers of surgery and the alternatives, at a time when reliable evidence is currently lacking.”

All best wishes, Dr Fiona Godlee, Editor in chief.

By return,

Dear Dr Godlee, I feel I owe you an apology. I am sorry your genitals were dragged into this debate. Best wishes, Laurence Shandy, gentleman.

The list of things we want is long. The meat of it is captured this week by another member of this august body, Richard Lehman in his journal review blog in the BMJ. He is often critical of industry and of the types of studies being published. Here is his very brief solution:

“All phase 3 trials to be designed and conducted independently of manufacturers, using the best available comparator. Research priorities to be determined by patients (James Lind Alliance). Value-based pricing. All data available from all trials, with meta-data: IPD [individual patient data] level for qualified independent centres. Big increase in comparative effectiveness research, much more research into non-pharmacological treatments.”²⁰

In conclusion, the evidence base is clearly flawed. Research is a human activity - we can’t expect perfection, but we are so far from perfection that there’s a great deal more to do.

I think what I and others have confronted increasingly, and are trying to come to terms with, is that there are so many systematic forces at work, pushing us in the wrong direction, and with vast amounts of money at their disposal, that we have a big fight on our hands. There are huge implications in terms of cost and waste and error, and human life.

The system is broken. I have reached a firm conclusion, and I am not alone in concluding, that there is a need to extricate medicine and research from industry. It will be a challenge.

1 McGauran N, Wieseler B, Kreis J, Schüler Y-B, Kölsch H, Kaiser T. Reporting bias in medical research - a narrative review. Trials 2010, 11:37 (13 April 2010). http://www.trialsjournal.com/content/11/1/37

2 Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in clinicaltrials.gov: a cross-sectional analysis. PLoS Med2009;6:e1000144. http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000144

3 Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012;344:d7292. http://www.bmj.com/content/344/bmj.d7292

4 Jefferson TO, Demicheli V, Di Pietrantonj C, Jones M, Rivetti D. Neuraminidase inhibitors for preventing and treating influenza in healthy adults. Cochrane Database Syst Rev2006;3:CD001265.

5 Kaiser L, Wat C, Mills T, Mahoney P, Ward P, Hayden F. Impact of oseltamivir treatment on influenza-related lower respiratory tract complications and hospitalizations. Arch Intern Med2003;163:1667-72. http://archinte.jamanetwork.com/article.aspx?articleid=215903

6 Doshi P. Neuraminidase inhibitors—the story behind the Cochrane review. BMJ 2009;339:b5164. http://www.bmj.com/content/339/bmj.b5164

7 Goldacre B. Bad Pharma. 4th Estate. London, 2012.

8 AllTrials http://www.alltrials.net/

9 Tamiflu correspondence with Roche. http://www.bmj.com/tamiflu/roche

10 Kmietowicz Z. GSK backs campaign for disclosure of trial data. BMJ 2013;346:f819 http://www.bmj.com/content/346/bmj.f819

11 Cohen D. Roche offers researchers access to all Tamiflu trials. BMJ 2013;346:f2157. http://www.bmj.com/content/346/bmj.f2157

12 Wieseler B, McGauran N, Kaiser T. Finding studies on reboxetine: a tale of hide and seek. BMJ 2010;341:c4942. http://www.bmj.com/content/341/bmj.c4942

13 Morton DL, Thompson JF, Cochran AJ, Mozzillo N, Elashoff R, Essner R, et al. Sentinel-node biopsy or nodal observation in melanoma. N Engl J Med2006;355:1307-17. http://www.nejm.org/doi/full/10.1056/NEJMoa060992

14 Torjesen I. Sentinel node biopsy for melanoma: unnecessary treatment? BMJ 2013;346:e8645. http://www.bmj.com/content/346/bmj.e8645

15 Kmietowicz Z. Drug firms take legal steps to prevent European regulator releasing data BMJ 2013;346:f1636. http://www.bmj.com/content/346/bmj.f1636

16 Cholesterol Treatment Trialists’ (CTT) Collaborators. The effects of lowering LDL cholesterol with statin therapy in people at low risk of vascular disease: meta-analysis of individual data from 27 randomised trials. Lancet 2012;380( 9841):581-590 http://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2812%2960367-5/fulltext

17 Abramson JD, Rosenberg HG, Jewell N, Wright JM. Should people at low risk of cardiovascular disease take a statin? BMJ 2013;347:f6123. http://www.bmj.com/content/347/bmj.f6123

18 Moynihan R. Scientists find new disease: motivational deficiency disorder. BMJ 2006;332:745.2 http://www.bmj.com/content/332/7544/745.2

19 Godlee F. Promoting cosmetic surgery BMJ 2012;345:e7535. http://www.bmj.com/content/345/bmj.e7535

20 Richard Lehman’s journal review—28 October 2013. BMJ. http://blogs.bmj.com/bmj/2013/10/28/richard-lehmans-journal-review-28-october-2013/