Currently women between 50 and 70 years are invited to present for mammography screening. The overall benefit of this is increasingly doubtful, as better research is published, but the information women are given does not reflect this. The present policy is based on quite old randomised controlled trials, more recent data being from observational studies which are less robust.

Successive governments have been committed to extending the age range for a long time. In 2010 the Department of Health committed to extending the age range at either end, so that women over 47 and up to 73 years would also be invited. The Cancer Epidemiology Unit in Oxford (headed by Sir Richard Peto) has for years argued for randomisation of the age extension roll-out in order to collect data on effectiveness. This converted the roll-out into a randomised controlled clinical trial, which is how it is described by the government.

The controversy over mammography screening centres on the possible harms. For any screening test that is not 100% specific or accurate, and if the disease concerned only affects a small percentage of people, there will usually be far more false positive results than real positives. That is very clearly the case for mammography screening. Hence more women have unnecessary surgery than have real cancers treated. The question is whether on balance this is worth the effort, cost, and harms of screening.

Issues of Research Governance and Good Clinical Practice (GCP)

The global standard for GCP is defined by the International Conference on Harmonisation Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH), although this is intended for medicines. However, as this is by far the largest clinical trial ever attempted, there is no obvious reason why best practice should not be followed. Here are some concerns that have been highlighted by questioning the sponsor and investigator.

1. What is the research question? If there is certainty of benefit (as initially assumed), there is no need for a trial. If there is uncertainty (also known as ‘clinical equipoise’), then the best way to test this is by a Randomised Controlled Trial, but best practice would include individual consent.
2. GCP requires that all trial personnel are qualified by education, training and experience to carry out their assigned tasks. The chief investigator has an arts degree and an honorary doctorate, and has refused to provide her CV and training record, despite repeated requests. She has however confirmed that she has undertaken GCP training, but gives no details.
3. No informed consent is being obtained. Women are assumed to have consented when they request screening. A standard leaflet is being provided to women invited, which does not clearly define the possible harms of screening.
4. There is little effective governance of the trial. The University of Oxford research governance head is responsible for about 1500 trials. We have not been able to find out who heads the data monitoring and steering committees, although we have been told that they exist.
5. The DoH says that the roll-out of the age extension will continue, irrespective of the progress of this trial. In other words, the testing of the proposed intervention, the extension of the age range, will not affect whether the intervention actually is implemented. They are testing the intervention at the same time as using it.
6. There does not appear to be any fully developed protocol, only a one-page synopsis on the website. We have not seen a statistical plan or data management process, or a quality plan. We made repeated requests to the Oxford University head of research governance, several senior people within Public Health England (PHE, of which the breast cancer screening programme is now a part), and the National Institute for Health Research. Professor Kevin Fenton, PHE National Director of Health and Well-Being, said that a full protocol exists, and that we should request it from the sponsor. We eventually received it after a third Freedom of Information request, and have subjected it to internal (within HealthWatch) and external review. Review comments are collated at the end of this document.
7. The primary outcome is breast cancer mortality. The sponsor says that the study is not powered to detect all-cause mortality, despite a planned trial population of 3 million. The Marmot review said that no previous trials were powered for all-cause mortality. It is obvious that all-cause mortality would be the ideal outcome, because the net benefit (or risk) is lives saved and deaths caused by the intervention. We have however learned that the protocol is to be amended to include the all-cause mortality analysis.
8. Marmot also recommended a cost-effectiveness analysis. Such an analysis has been completed, and this concludes that “The NHS breast screening programme is only moderately likely to be cost effective at a standard threshold”. However no action has been taken as a result of this.
9. The trial is funded by the Department of Health, who also originally employed the chief investigator (under the reform of the NHS, the cancer screening programmes have been transferred to Public Health England). But she is a visiting professor at Oxford so is using that as her affiliation. In industry it would be unknown for an investigator to be affiliated to the sponsor, but this apparently is normal in academia. Nevertheless, the chief investigator is primarily the national director of the Breast Cancer Screening Programme, a position which is clearly in conflict with her role as an impartial assessor of the evidence for or against screening.
10. The age extension RCT is justified by the National Screening Committee on the basis of a pilot study to assess the feasibility and acceptability of rolling out screening in these age groups using a cluster randomised design. The authors of this pilot study concluded that “No major problems of feasibility or acceptability of randomization were found”. Among a range of serious methodological problems, the study did not collect any data on acceptability by patients. A critique of the study has been posted on PubMed Commons.

These would be serious issues for an industry-sponsored trial. The Medicines and Healthcare products Regulatory Agency (MHRA) would not tolerate such a lack of governance. The National Institute of Health Research says that the study carries such low risk that only minimal monitoring is required. However here we have not only the largest clinical trial ever attempted, funded by the public purse, but it is being set up to support a public health policy for which there is doubtful evidence. Such a research programme should surely be subject to the utmost transparency, and be conducted to the most rigorous standards of quality and governance. But simple questions go unanswered, and oversight appears inadequate.

Review of the Protocol

Introduction

Over a period of about six months HealthWatch members have tried to obtain the full protocol for this trial. We made requests to various parties including the chief investigator, the sponsor, the ethics committee, and the National Institute for Health Research. We were repeatedly referred to the summary online at http://www.controlled-trials.com/ISRCTN33292440. This of course is only a synopsis and lacks the necessary detail to enable anyone to implement the trial.

Only after three Freedom of Information requests did we obtain the allegedly approved Protocol from the sponsor, the University of Oxford.

We (LR, SB) reviewed the protocol internally, and then invited eight other experts with a variety of expertises related to clinical trials to give us their private comments1. The following is an anonymised summary of the comments, with key statements.

It should not be assumed that all reviewers can individually be associated with all of the comments collated here. While there was remarkable consistency, reviewers placed different emphasis on particular issues.

Overall Digest of Reviews

All reviewers commented on the brevity of the document, only eight pages for what is claimed to be the largest randomised controlled trial (RCT) ever attempted. Only two references are given to support the text. The lack of transparency, e.g. no details on governance committees, and difficulty in obtaining the protocol, were thought to be suspicious for a public sector trial.

Reviewers noted that the protocol is called version 1. No revision history is given, and it is almost inconceivable that this version was issued without any prior drafts. The sponsor insists that the protocol was peer reviewed, but we do not know by whom or when. Without any revision history it is impossible to verify that there was any peer review. In industry it is virtually unknown for any clinical trial protocol to be totally free of amendments; yet this is dated November 2009, received ethics favourable opinion in February 2010, and has remained unchanged throughout despite the emergence of new and relevant scientific information.

Surprise was expressed at the credentials of the chief investigator. A history degree was not considered appropriate for a person leading any clinical trial involving human subjects, let alone one as large as this. The chief investigator is thus not answerable to a health regulator.

The trial aims were described as “woolly”. There is no clearly specified primary objective relating to a testable hypothesis.

The rationale shows that the government has decided to carry out the intervention (i.e. extending the age range) before the trial results are available. Also in this section the capture of data on harms is very unclear, with no detail on how such data will be handled.

Most reviewers were extremely wary of trials that dispense with individual patient consent, although a couple recognised that cluster randomisation can be ethical, and that testing of policy implementation could be considered ethical.

In this case it seems that researchers who are not involved in the patients’ care will get to see details about them, which violates a normal principle of consent.

The assumption that the study would be `widely known’ in a community because of inviting women for screening was considered far too optimistic. Women in the uninvited batches cannot understand they are taking part in a trial. Also, it could not be assumed that women in the invited batches would understand they were taking part in a trial. It is unclear whether women eligible for the 47-50 extension could opt out of the trial, but still be eligible for screening when they reach 50, or whether they will be subject to the usual systems of follow-up, pressure to have screening and signing of ‘disclaimers’. The GPs of women in the trial are not being told about it, which it was thought could bias recruitment.

There is no sample size calculation, despite statements from the sponsor that the trial is not powered to assess all-cause mortality. One statistician in the review group calculated that it should be possible to detect this, given a population of three million women as planned. He said that the limited text on data analysis was meaningless without a sample size calculation. Another reviewer considered that while a formal sample size need not be stated, there should be a power calculation. The investigators do not state what estimate they are taking for the effect on breast cancer mortality of the current (50-70 years) programme and what is a clinically important difference in mortality. The method of statistical analysis is not stated.

The methodology is imprecise. For example, it is stated that screening batches will contain 200 women each, but no number of batches is given. There is no provision for a stopping rule, if the trial either shows clear benefit or serious harms early (but they are not measuring harms in any serious way). There is no economic analysis, which surely should be a secondary outcome for a publicly funded programme. Women randomised to the control group can still request screening if they are over 70 years, or if 47-50 and they live in an area covered by the trial. This was thought to risk contamination of the randomisation.

All-cause mortality would be a more objective endpoint. The use of cause-specific mortality means that it is important to be more than usually certain of the cause of death, but no procedures are specified to verify causes of death.

The proposed analysis comparing invited and uninvited batches was thought to be potentially problematical, by introducing subtle biases. For example, could getting in touch with women in the invitation group mean that they are less likely to be lost to follow up by end of the study?

There is no explicit description of data anonymisation. This has been found in other public sector studies to have been inadequate (e.g. the UK BioBank study).

Other key omissions were pointed out, including:

1. No endpoints stated up front (not just buried in the Analysis section)
2. No clear start and stop date of the study
3. No indication of number of patients to be enrolled (or a statement that there is or is not an upper limit of ‘n’)
4. No clear methods – especially a detailed description of how procedures with ethical implications will be handled
5. No detail on data management
6. No statistical section describing the tests to be used and who will do the analysis
7. No publication policy.

Curiously the protocol includes a timetable or task list, i.e. there were at the date of this version still several essential documents and other deliverables to produce. One of these is “Establish mechanism for data download from NBSS system”. In other words, the authors did not know how data management would be done. This task was scheduled to be completed after starting the trial, i.e. at start-up the investigators did not know whether and/or how they could capture and process data. In this context, there is no information on how certain key data will be flagged. For example, women with prior breast cancer will be excluded from analysis, but how these records will be flagged and filtered out must be specified up front. If it can’t be done then the trial can’t start.
Some key verbatim quotes from invited experts:*

“Are they seriously claiming that’s the full protocol? That’s rather scary”.
“The stated rationale shows that we’re looking at policy-based evidence here, not evidence-based policy.”
“Assumption that all women in routine care will permit for their data to be used in this way is naive and wrong!”
“This is not a research protocol. This is no more than a statement of intent.”
“Why is the government taking advice on patient care from a non-scientist?”
“Why is this person, who will not share proof of GCP [good clinical practice] training (confirmed by a test or certificate of some sort) in charge of research on humans?”
“Why is the EC [Ethics Committee] approving something like this?”
“It’s fair to say that by not looking at rates of mastectomy and radiotherapy, the study as presented does not give a fair view of the disadvantages as well as the advantages of screening.”

This review leaves the following questions unanswered:

1. Who has oversight and the power to stop the trial if it is scientifically or ethically wanting?
2. What are the memberships of the Trial Steering Committee and the Data Monitoring and Ethics Committee, and where are their minutes?
3. Was the protocol reviewed in the light of the Marmot report?
4. Are the investigators in equipoise or not (whatever they were at initiation)?
5. Should the trial be halted?
6. Should it be modified so that it becomes individualised consent?

Les Rose
Susan Bewley
20th July 2014

* The experts included several professors, eminent epidemiologist, expert in evidence based medicine, clinical trials manager, medical statisticians, and freelance consultants in clinical research. Clinical disciplines include primary and secondary care and midwifery, and one member of a research ethics committee.