Sounds a good idea? For wealthy pharmaceutical companies, perhaps, who can fund the extra costs through sales of the drug being tested. But what about non-commercial trials who want to test methods of treatment for which they do not stand to gain financially? There are now desperate protests from academic researchers who claim that the Directive will sound the death knell for unbiased research. Others, however, believe the problems can be ironed out by discussion. Meanwhile the general public, whose vague understanding of research methods is coloured by images of human guinea pigs remains confused or, more commonly, indifferent.
This method of finding out about the world is, on the whole, fairly simple and valuable. But it can also be profoundly fallible. The bothersome fact is that some things dont follow a consistent logic. Flames might always be hot, but dogs are not always friendly. Relying on our own experiences of life can be dangerously misleading. It is the source of much misunderstanding and prejudice.
This is why science has come up with ways of reducing mistakes. The methods are not foolproof, and they take time and skill, but they cut down the likelihood of serious error. They help us make sense of events that are variableand of truths that are counterintuitive.
Most of us are resistant to being told our common sense is wrong: if we have seen something with our own eyes it must be true, and we naturally prefer to learn from anecdotes than from statistical calculations. Thus, though I and other wiser people have campaigned for twenty years to promote wider understanding of scientific methods of assessment, we have failed. And in particular, we have failed to popularise the notion of randomised controlled trials.
The very term sounds as if it is a subject for sad anoraks who need to get out more. But randomised controlled trials, or RCTs, are key to improving health and longevity, and our failure to know about them and apply them properly undoubtedly results in the unnecessary deaths of thousands of people every year. Understanding RCTs is almost as important as understanding that flames are hot; without that appreciation you can get badly burned.
In simple terms, RCTsor fair tests of medical treatments as I have suggested we might rename themreduce the influence of the two great contaminating features of personal experience: bias and chance.
Take chance. It is easy to underestimate how apparently powerful effects can simply be coincidence. Suppose I want to know if a new drug speeds recovery from surgery. If I treat some people with the drug and they all thrive I might assume the medicine is beneficial. But how do I know they would not have got better that quickly anyway? How do I know whether they would have recovered faster without the medication? Suppose one of them deteriorated or died. How would I know whether I should blame the drug?
The only reliable way to tell is to invite patients to participate in a fair test by dividing them into two groups using random allocation, and then giving patients in only one of the groups the drug. We would then measure whether they recovered faster than patients in the other (control) group. And if a sufficiently large number of patients participate in this way, we can reduce the likelihood that we will be misled by the play of chance.
But we also need to guard against other sources of bias.
Humans are not dispassionate machines; we are prone to hopes and fears, expectations and disappointments. As Aristotle noted, people are easily deceived, for example: the coward when excited by fear and the amorous person by amorous desire; so that with little resemblance to go upon the former thinks he sees his foes approaching, the latter that he sees the object of his desire. Expectations create bias. This is why some quack potions work for those who believe in them even when the remedies contain no usefully active ingredientswhats known as the placebo effect. Indeed, if you and I are each given a pill, and you are told yours is an experimental cure for headaches while I am told mine is just chalk, I will be predisposed to thinking my pill will be a waste of time and may be less likely to experience any benefit. In well-conducted trials, therefore, to guard against these biases, participants are not told such details. These are known as blind trials.
It is not just participants who can be biased. Hundreds of studies have shown that researchers themselves can hugely influence the outcome of trials, sometimes deliberately but usually unconsciously. To try to minimise this risk, those who conduct the procedures and the measurements are also kept in the dark about the purpose of the experiment. When steps are taken to minimise bias among all those involved the study is known as a double-blind trial.
OK, it all may sound a bit of a palaverand thats why most members of the public are ignorant about the methodology, and why so many clinicians, patients and policymakers prefer to rely on what they may see as common sense rather than the results of controlled experiments. But the result of taking short-cuts is that much of our orthodox medicine is wasteful, and some is downright dangerous. That is why this edition of the HealthWatch Newsletter now takes up the cudgels.
Nick Ross
Writer, broadcaster and president of HealthWatch
But when I graduated I discovered that this principle was not easy to apply. I worked in a unit in Jamaica where severely malnourished children were admitted to our ward, and often soon died, despite receiving the recommended treatment. They were far below normal weight for age, with wasted muscles and wrinkled skin, so obviously they needed to be generously re-fed and rehydrated, but they died anyway. For reasons explained elsewhere [1] we tried giving much less fluid and protein than textbooks advised, and observed that fewer children died: the commonsense treatment was killing more children than it saved.
In the 1950s similar unexpected effects were being observed in many research units in different countries. For example, in California premature infants in incubators were (of course) given supplemental oxygen, since their lungs were immature and inefficient. Many of them unfortunately developed retrolental fibroplasia, that caused them to be blind. After many years of research it was revealed that it was the excessive use of oxygen that caused the blindness [2]. In retrospect this research is seen to be valuable, but at the time the researchers were accused of reckless innovation. In some instances this criticism was justified [3]. Over the last 20 years the design of randomised controlled trials (RCTs) has been greatly refined, and so also have the regulations to ensure better protection of the participants.
So where is the Primum non nocere principle now? Virtually every type of treatment carries some risks that must be balanced against the potential benefit. Drugs (even natural herbal drugs) have side effects, surgeons must cut through healthy tissue to reach diseased tissue, psychologists recovering memories of childhood trauma may do harm as well as goodthe list is endless. A well-designed RCT (when that is possible and ethical) will measure the benefit/risk ratio for a given treatment. Contemporary RCTs are still discovering plausible and well-intentioned treatments that do more harm than good: for example, strict bed rest is no longer good treatment for angina or arthritis, or albumin infusion for hypoalbuminaemia, or hormone replacement therapy to prevent hip fractures.
The media are still prone to criticise the ethics of clinical research, but the burden of ethical justification is changing. Since we now know that so many evidently benign treatments have unforeseen dangers, the clinical researcher may reasonably ask the good, honest practitioner to justify his traditional treatment. Primum non nocere applies just as forcibly to both.
References
Doll, whose most famous research revealed in 1950 that smoking causes lung cancer, was an early pioneer of the randomised controlled trial. During a discussion following a lecture in Oxford in February, he told an audience of 200 doctors and students that he felt the only way to highlight the magnitude of the current threats to patient-orientated research would be to deliberately flout the regulations and be punished in consequence, possibly with a prison sentence, and that he would be prepared to do this.
Doll later told the HealthWatch Newsletter that he stands by his statement. The Clinical Trial Directive 2001/20/EC is, he told us, just the latest example of a process that has been going on for years. The growth in the number of regulations, for example restrictions on access to medical records, is seriously hampering medical research and it needs to be brought to the publics attention. Unfortunately I think the only way to do that now is for someone to break the law and go to prison.
Now aged 91, and with over 500 published studies to his name, Sir Richard Doll still works daily at Cancer Research UKs unit in Oxford. His original comment was made after a lecture entitled, Over-regulation of clinical research is damaging the public health given by Professor Charles Warlow of Edinburghs department of Clinical Neurosciences, which took place on 20th February at the Witts Lecture Theatre, Radcliffe Infirmary, Oxford.
Reform or repeal the Clinical Trials Directive, advise researchers from Exeters Department of Complementary Medicine, before clinical trials become an endangered species. Dr Peter Canter and Professor Edzard Ernst of the Peninsular Medical School speculate on Directive 2001/20/ECs implications in the March issue of the Journal of the Royal Society of Medicine (available to JRSM subscribers on http://www.rsmpress.co.uk).
The Directives aimsharmonising regulation and protecting public healthare worthy but, they suggest, unfortunately those who made the rules were preoccupied with commercial research, at the expense of academically led clinical research of the sort to which we owe many of the big advances in medicine. The resulting extra layers of bureaucracy could add a year to the preparatory work involved in planning a trial, involving costs that may not be available in the case of academically-led research, and could even insert a Catch-22 situation whereby applications might not be approved unless they have already secured funding, while funding might not be secured until applications have been approved.
J Roy Soc Med 2004; 97 (3): 1012.
If studies are not done properly, any results they produce will be worthless. We call this validity. So begins Quality and Validity, an essential piece of intelligence produced by Andrew Moore, editor of Bandolier, the website for evidence-based thinking about health care. Moore has authored Bandolier Professional, a series of articles explaining key principles of research techniques. They are freely downloadable from the following link: http://www.msdforphysicians.co.uk/bandolier/bandolierteaser.asp and are recommended reading. The home page for Bandolier, also recommended, is http://www.jr2.ox.ac.uk/bandolier/
The articles are downloaded as Acrobat .pdf files. If you do not have a copy of the Adobe Acrobat reader then you can download it from here
Informing the public is always a good idea in principle, but often problematic in practice. It is sad that none of the organisations purporting to educate the public about science appears to have made any sustained effort to help people understand the difference between treatment claims based on randomised experiments and those based on non-randomised, observational data. The print and broadcasting mass media dont help. Those who shape the content of the media often appear to regard public education as a secondary objectiveif indeed it is an objective at all. Editors want stirring stories and provocative features. The main writers and broadcasters are not expected to be experts; their job is to produce stimulating copy. This can lead to woeful misconceptions when it comes to coverage of randomised clinical trials.
For example, in an article entitled Random clinical trials are one of lifes biggest gambles, the influential journalist Polly Toynbee concluded that randomised clinical trials should be abandoned. It may be a little less accurate scientifically, she wrote, but if patients are allowed to choose which treatment they want and every detail of their condition, lifestyle, character and circumstances is fed into the trial data, I doubt if the results would be seriously distorted. And she made clear her reluctance to agree to be a guinea pig [2].
Terms such as human guinea pig and experimentation are hardly neutral. They conjure up mental images of hapless patients satisfying the intellectual curiosity of researchers, if not Nazi concentration camps and the Nuremburg Trials. And yet these images are promoted even by publications purporting to represent the interests of patients. The cover of the December 2002 issue of Health Which?, for example, featured the headline: Guinea pigs: The risks of volunteering for clinical trials.
A few journalists do seem to understand the issues. Nick Ross, for example, demonstrates this amply on page 1 of this issue of the HealthWatch Newsletter. But he and other informed broadcasters and journalists have to battle against the stereotypes and the ready clichés favoured by their colleagues. Take, for example, an article published by The Guardian in 2001. The report of a controlled trial assessing the effects of transplanting fetal tissue into the brains of people with Parkinsons disease [3] prompted the newspaper to commission a freelance health journalistSophie Petit-Zemanto write a piece setting the study in the context of a general discussion of clinical trials. I saw the manuscript she submitted, and thought it was excellent. However, the way it was subedited is an illustration of the way that market share can trump public education as a driver of editorial practice [4].
The original title of the Guardian articleClinical Trialswas changed to Trial and Error. Although the manuscript had made no reference to human guinea pigs, a subtitle reading Medical research needs human guinea pigs was added to the piece, and a sentence at the end of the edited first paragraph read: How do you assess potential risks when the whole point of the treatment is that you are the guinea pig? The original piece had used the word experiment once; the subedited version used it three times in the first four paragraphs. Alongside a photograph captioned Experiment a surgeon working on the failed Parkinsons trial, the subeditor decided to add that a trial of a new treatment for Parkinsons disease had, for some patients, gone horribly wrong, and that the researchers were horrified because the cells appear to have gone into overdrive, producing too much dopamine and causing the patients to writhe and jerk their heads uncontrollably (my emphases).
This example of an editor at a quality newspaper distorting and compromising a diligent journalists attempt to improve public knowledge about controlled trials is instructive. It suggests that those who control the content of the print mass media feel that sensationalisation should override efforts to educate the public.
Carolan Davidge, head of the press office at the Medical Research Council, asserts that media research does not support perceptions among clinical trialists that the media regularly refer to human guinea pigs or experimentation, and she believes that media coverage of clinical trials is predominantly positive and accurate [5]. I have urged her to publish the methods used in this research and the findings, so that it becomes possible to assess its likely validity. If appropriate, the findings will begin to influence the perceptions of those endeavouring to promote clinical trials. Until then, impressions and illustrative anecdotes such as those used above will continue to shape opinions.
I failed in my attempt to engage Polly Toynbee in a discussion of the article cited above, but her views left me wondering whether she was aware that randomised trials tend to yield different results from statistically adjusted non-randomised studies addressing the same questions [6]. On the basis of the statistically-adjusted non-randomised studies that she favours women have been told over a couple of decades that taking hormone replacement therapy (HRT) will reduce their chances of getting heart attacks and strokes. Randomised clinical trials, by contrast, suggest that HRT increases the risks of these serious complications of cardiovascular disease. If she is consistent, I suppose Ms Toynbee must continue to believe that HRT reduces a womans risk of heart disease and stroke.
Iain Chalmers
Editor, James Lind Library (http://www.jameslindlibrary.org)
References
The Save European Research campaign (http://www.SaveEuropeanResearch.org) was launched on 9 December 2003 out of concern for the future of academic and investigator led research in Europe with the advent of the European Union Clinical Trials Directive. It was launched from a cancer platform but is inclusive in spirit and has already gained momentum in cardiology, dermatology, and psychiatry.
The directive was written and passed after minimal consultation with interested parties in the member states. Frustration at the inevitability of its arrival in May 2004 and the lack of a process to alter its course was much discussed at medical conferences and meetings around the world last year. This concern is reflected in the fervour with which researchers from all over Europe and the world have signed the letter to MEPs on the campaigns website (reproduced below). It starts, why did the European Union decide to stop cancer research, and by 15 January more than 2000 researchers had signed.
The directive raises the bar in terms of quality and reporting standards for all research. The pharmaceutical industry has understandably accepted this as an achievable minor inconvenience. What is new is that this is now the minimum standard for all clinical research regardless of the financial backing or goal of the project. Under the directive all investigators must take on more paperwork, liability, reporting, and cost burden.
Little was wrong with the processes of academic or investigator led research in the European Union in the first place. Many important medical breakthroughs in recent times have been a product of this mechanism. Several eminent American and Australasian researchers have signed the letter, with messages of support that they consider it bad news for the development of medicine if European academia is shut out.
To be updated on the progress of this campaign, please sign the letter and provide an email address to receive regular reports.
Brian Moulton
Chief Executive Officer , Irish Clinical Oncology Research Group
120 Pembroke Road, Ballsbridge, Dublin 4
email: brian.moulton@icorg.ie
Reference
Moulton, B. Save European Research. British Medical Journal 2004; 328: 286.
(31st January 2004)
Dear MEP
Why did the European Union decide to stop cancer research?
This is the question which European Parliamentarians, Commissioners and officials will be asked by their constituents in May 2004. On that date the European Commission Directive (2001\20\EC) on clinical trials comes into force. This directive places such high administrative expenses in the way of patient-focused research, that it will effectively end all clinical research except for those trials which are commercially-inspired, and drug company-sponsored.
The bulk of cancer research trials are currently conducted, not by the industry, but on a voluntary basis by cancer specialists and charities, who simply do not have the resources to meet these new requirements. Thus, trials which are vital to the best interests of patients, but are of no interest to the pharmaceutical industry, either because they involve generic or widely available drugs, or (as is the case in trials of screening, radiotherapy and surgery), no drugs at all, will be nearly impossible to conduct. Investigations of new treatments for those rare fatal cancers which affect children may stop altogether.
If this directive had been introduced forty years ago, many of the most critical advances in cancer treatment would not have been made. Women with breast cancer would still have to lose their breasts, and patients with throat cancer their voice boxes. Childhood leukemia would still be a death sentence rather than a great success story of cancer research.
The directive represents a solution which is not needed to a problem which does not exist.
It is no easier for organizations than it is for individuals to admit that they have made a mistake. The EU had made one here, and public pressure will force its correction. The question is whether it will be corrected now before it costs lives, or later, after it has.
We respectfully urge the European Parliament to repeal this directive before it comes into force.
PLEASE CHANGE DIRECTIVE 2001\20\EC BEFORE ITS TOO LATE
There will be new tricks to learn, but the benefits include:
It is never easy to deal with change, but the suggestion that the Clinical Trials Directive marks the end of academic clinical research in the UK seems hard to support; examples given seem to be part of a looking glass war. It should be remembered:
Think back to the system of Crown Immunity (and its exploitation by kitchen staff to prevent inspection of dirty kitchens); self-insurance (meaning no insurance); retention of organs without consent; clinical trials that were an abuse of patients rights and sometimes harmful; fabricated clinical trial results upon which major decisions about treatments were based; and the casual Chairmans approval for protocols the rest of the Ethics Committee had never seen and could not find on their files when things went wrong. Think of the way we have improved and you will see that the benefits that will accrue with the implementation of the directive are commensurate with the effort that will be required to implement it.
No, the Clinical Trials Directive does not mark the end of academic clinical research in the UK. Research will be more difficult to perform but better in quality: this fits in pretty well with the way of the modern world. A characteristic of the UK is a lack of consistency such that in everything we can be both the best and the worst. In research this cannot be regarded as engaging eccentricity. If all physicians were meticulously careful and honest, there would never be the need to check anyone. But the record shows that whenever checks are made, mistakes are found; and fraud is not always far behind.
Michael E Allen
Regulatory Consultant and Honorary Secretary, HealthWatch
The volunteer does not know (is blind to) the contents of the capsule he is taking. So also is the investigator who measures the response to the drug (so the design is double blind). The trial is controlled because the response of the volunteers on drug A is statistically compared with that of volunteers on B (the control) treatment.
Note that people in clinical trials are volunteers: it is unethical (and illegal) to recruit to a trial a person who has not given informed consent. The investigator must ensure that everyone understands what the trial involves, what question it is intended to answer, and what danger or discomfort it is likely to involve. No sane person will agree to be randomly assigned to treatment A or B if there is already good evidence that one treatment is less effective, or more dangerous or uncomfortable, than the other. Because this is a judgement that the average patient cannot make with confidence, every clinical trial involving human volunteers is reviewed by an Ethics Committee which has this knowledge, but which is not involved in the conduct of the proposed trial. They decide if the design of the trial is likely to provide an answer to a clinically relevant question, and if all possible steps have been taken to protect the volunteer from danger and discomfort. For these requirements to be met investigator and volunteer must trust each other.
Unfortunately, as Iain Chalmers reports, the media sometimes seem intent on destroying this trust by implying that investigators treat volunteers as human guinea pigs, whose autonomy and safety are of little concern. To guarantee the autonomy and safety of volunteers many regulations have been imposed to ensure that the highest standards of clinical care and scientific honesty are maintained in the conduct and reporting of results of clinical trials. But strict legislative control is a double-edged weapon. Imposing ever stricter controls on investigators does not guarantee a greater output of excellent trials. Indeed, the controls may become so burdensome that academic investigators are unwilling to conduct trials at all, so the effect of the controls is the opposite of that intended. Where is the best balance between controls being too weak to ensure good clinical practice, or so bureaucratic that clinical research is stifled? Below I give personal answers to some critical questions. I am most grateful to Michael Allen, Iain Chalmers, Sarah Meredith and Charles Warlow for helping me to understand these problems, and suggesting possible solutions.
Between the years 1995 and 1999 the average annual number of clinically-oriented RCTs sponsored was 90, 20 and 20, for NHS, MRC, and charities respectively. In the years 2000 and 2001 the number fell progressively. In 2002 it was 21, 7 and 11 respectively for these three main non-commercial sponsors, and most were drug trials [1]. It is ironic that, when the Government is stressing the importance to health of lifestyle factorsin particular smoking, diet and exercisetheir support for trials to establish the validity of such advice has almost vanished. It is not that this lifestyle advice is already underpinned by solid evidence. To my knowledge there is no compelling evidence that increasing exercise and decreasing intake of sugar and fatty food is the best way to control the epidemic of obesity: that may be true, but it has not been proved.
It matters very much indeed. There is clear evidence that pharmaceutical companies bias the reporting of drug trials in favour of their own products (see the theme issue of BMJ, 31st May 2003 and this Newsletter, issue 50, July 2003). It is therefore essential to have independent trials to provide reliable evidence about the efficacy of drug treatments.
No. The arguments for and against the Directive are set out on the centre pages of this Newsletter. Dr Moulton claims Little was wrong with the processes of academic or investigator led research in the European Union in the first place, which is comforting, if true. But issue 52 of this Newsletter carries the uncomfortable message from Dr Wilmshurst that research misconduct does occur in distinguished academic medical institutions, and is very difficult to expose when these institutions refuse to confront it. Will the Directive stifle important academic research? It may do, depending on how the MHRA, the regulatory body in the UK, interprets the rules. Dr Sarah Meredith, of the MRC Clinical Trials Unit, has given examples of trials that would be almost impossible under the rule that all medicines used in trials must be specially labelled [2], but the MHRA responds that such difficulties can be overcome by discussion [3]. Both sides agree that the Directive will increase the documentation and expense of trials, but, as Dr Moulton admits, it will raise the bar in terms of quality and reporting standards for all research. One reason it will make trials more expensive is that sponsors will be held liable for any mishaps that affect participantsin other words, they will have to take out insurance. Is this a bad thing? We need volunteers to participate in trials so the standard of healthcare we receive can be improved. It is their civic duty, isnt it? So isnt it our civic duty to ensure that, if they suffer mishap in a trial, they can receive appropriate compensation? The cost of insurance will be proportionate to the risk involved.
This question was brilliantly answered by Professor Charles Warlow at a meeting at the RCP Edinburgh on 31st October 2003. When it is published we will provide a link from the HealthWatch website to the full text on http://www.bmj.com. My summary of the main obstacles are as follows: for his details and documentation read the full paper.
Research ethics committees
There is an inherent contradiction about the membership of local research ethics committees. Obviously they should not be composed entirely of experienced research workers, since the committee exists to curb the excessive enthusiasm of researchers. But does absence of first-hand knowledge of clinical research make someone a truly ethical person? Furthermore, it seems that ethics committees search diligently for something they can forbid researchers to do, even if that then makes the research impossible. This situation may improve under the Clincal Trials Directive if ethics are monitored by competent professionals.
Who pays for what
A decade ago there was a gentlemens agreement that sponsors paid for the research component of trials in NHS hospitals, while the service component was covered by the NHS. This is now unacceptable, and the impossible task of calculating exactly the relative cost of the two components causes serious delays.
Personal data protection
The Data Protection Act (1998) is probably an important cause in the decline in non-profit trials from 1999 onwards. It is a minefield for epidemiological researchers who wish to review patients medical notes. Curiously, administrators (seeking to run the NHS economically) are given a free pass through this minefield, even in areas which researchers (seeking to provide better healthcare) enter at their peril. It seems that audit is commendable and necessary, but research is a dangerous luxury.
Adult incapacity regulations
Professor Warlow is involved in the treatment of patients with brain injuries, from stroke or trauma. Such treatments work best if given as soon as possible after the injury. He is understandably incensed that, if the patient is unconscious, he must wait (usually for an hour) while a proxy is found to consent to use of an experimental treatment being subjected to an unbiased clinical trial. If he arbitrarily chose a treatment he could give it immediately.
Diagnostic samples and post-mortem tissue retention
Legislation in this area was hastily drafted in response to public fury about organs retained without consent at Alder Hey hospital. The resulting law is incredibly perverse. Any tissue sample (taken with permission of the patient) must be destroyed when the specific test for which it was taken has been done. To archive sera for future research is illegal.
Imagine the effect if comparable rules applied to cartographers. All existing maps and Ordnance Survey data must be destroyed, and any future survey must be to facilitate a specific journey, and when that journey is complete those data must be destroyed! But the parallel with medical research is quite close. We inherit from past research a framework of understanding in a particular area: that is the map on which future research is planned, and archived material may be invaluable in checking that we are actually making progress.
Independent clinical trials, on which improvement of healthcare depends, are in mortal danger for three reasons. First, the public do not respect or trust trialists. Second, the government sees little need to support such trials either morally or financially. Third, legal safeguards designed to protect the autonomy and privacy of patients are in fact exposing such patients to the dangers of inadequately tested therapies.
Solutions to the problems are not easy, but here are some suggestions. Journalists who describe volunteers as human guinea pigs should consult their consciences. Legislators should recognise that clinical researchers are no less honourable than health service administrators, and should be allowed equal access to patients records. Non-profit trial sponsors must be willing to raise the standard of quality and reporting of their research, accept liability and insure their volunteers. The MHRA must interpret the Clinical Trials Directive so as to strike the right balance between reducing bureaucracy and protecting patients. Legislators and electors must realise that Draconian controls do not necessarily produce more patient-friendly research, or a better health service. We have now reached a point at which some of the best minds, who would have opted for an academic career, note the tribulations associated with clinical trials, and opt for less oppressive employment. Because trialists will be so burdened with rules, made by well-meaning legislators to address quite different problems, there is a real danger that that independent clinical trials will cease entirely. For the sake of good healthcare this disaster must be averted.
John Garrow
Chairman of HealthWatch
Emeritus Professor of Human Nutrition, University of London
References