by Stephen Bradley
The work of patient activists, campaigning clinicians and researchers in the field of meta-research (‘research on research’), has been crucial in exposing the crises in medical research. Achievements like mandatory registration of clinical trials and the increasing popularity of open science practices show that change is possible, but we still have a long way to go in making research more transparent and rewarding academics for genuine discovery rather than obtaining ‘positive’ or newsworthy results.
Without promoting clear and achievable remedies to these problems our mediocre status quo will persist. Although many fantastic initiatives are having a real impact on improving research culture we cannot rely on these efforts alone. We should not tolerate a situation in which the basic documentation that is necessary to scrutinise publically funded research is frequently unavailable and there is still no straightforward way to find out about researchers’ conflicts of interest. Whilst there are basic measures that could improve this situation we must surely demand their implementation.
Just over two years ago some of the participants at the Evidence Based Medicine Live Conference came together to advocate for this change. We believe that the systemic nature of the problems in research along with the scale of the resulting harms represents a challenge which the entire public, not just motivated researchers, have a stake in resolving.
Through our Declaration we proposed three actions that warrant immediate implementation:
- Mandatory registration of interests for all health researchers;
- That journals allow researchers to publish using Registered Reports, whereby peer reviewers make decisions based on study methods rather than the results;
- That all publicly funded research is pre-registered and published on a research registry along with basic documentation such as protocols.
These measures are not intended to be a ‘gold standard’, but represent basic minimum standards patients and taxpayers are entitled to expect, right now, from research.
HealthWatch was one of our first signatories, an important early milestone for the campaign and we remain extremely grateful for the support and suggestions we have received from many HealthWatch members. Over 100 other individuals and organisations worldwide have since also signed the Declaration. Maintaining an international perspective is important to us and signatories have translated the Declaration into Arabic, German and Portuguese. We are grateful for support from organisations including the German EBM Network and the Oxford-Brazil EBM Alliance along with collaboration with the US Based Centre for Open Science.
We have authored several articles, including in BMJ and The Conversation. Our review paper, which was conceived both as a primer on the principal problems in research and to explain how the aims of the Declaration could help address these, has been accessed over 2,000 times and has been cited by other important work in the area.
Signatories have presented the Declaration at the REWARD | EQUATOR Conference in Berlin and the Wellcome Trust’s Reproducibility, Replicability and Trust in Science conference. Our twitter account @ImproveHealthR has almost 600 followers and our website ImproveHealthResearch.com went live last October. In April this year we collaborated with the Centre for Open Science for our first ‘hack-a-thon’, in which participants emailed the editors of dozens of the major medical journals and asked them to consider adopting the registered reports format.
A mountain to climb
Our vision for a basic charter that succeeds in uniting patients, researchers and clinicians in a high-profile campaign remains unachieved. Few academics and even fewer members of the public will even have heard of the Declaration. The unprecedented public engagement with science during the pandemic has helped us to explain the problems and the need for change, but the polarisation of the past 18 months has also prompted reflection on the challenge of pressing for urgently needed reform without engendering further division. We have also not yet managed to get our voice heard in the mainstream media.
In the next two years I think our strategy should prioritise:
- Reaching more of the public and patients, including through the mainstream press;
- Engaging directly with policy makers and funders;
- Establishing new collaborations with other campaigning organisations, and developing existing collaborations such as those we have with HealthWatch and the Centre for Open Science.
The responses we have received to the Declaration have been overwhelmingly positive, but we are very grateful to those who have taken the time to explain why they disagree with us. The following are condensed summaries of many responses received and are inspired by no specific individuals. The nuance of the many discussions that we have had cannot be conveyed in a concise way and these composites are necessarily somewhat caricatured.
“What’s the problem?”
Surprisingly many academics seem largely unware of the problems, or feel that talk of a ‘replication crisis’ is over-dramatic. Sometimes we hear that “all those problems have been fixed now”, invoking for example, clinical trial registration, although this does little to prevent abuses like ‘outcome switching’ and neglects the fact that clinical trials are only one type of health research. We sometimes receive comments that misunderstand our position, for example, that that we seek to prohibit exploratory research or that we are calling for data to be shared that would lead to violation of patient confidentiality.
“It’s too little, too much, or not what I would have chosen”
It is certainly true that even if our demands were implemented in full, many problems would remain. For example, researchers could still commit outright fraud. However, some have commented that what we propose would be too disruptive, while others have argued that what we propose is both asking for too much and that our proposals would not improve matters enough. Our focus on just three demands required difficult decisions and a few have been disappointed that we did not include other objectives. In responding to these concerns we have tried to express our goal of raising basic standards across the research system. This necessitates embracing ‘better’, even though it is a long way short of ‘ideal’. When considering whether to support us the most pertinent question is not "are these three measures what I would have chosen?" but "could these measures improve the matters, compared to what we have now?"
"It’s not fair to change how we do things because of a few bad apples”
This perspective sometimes comes from institutions and tends to emphasise the responsibility for (other) individuals to improve standards. This is informed by a sense that problems in science are predominantly down to wilfully dishonest and negligent practices. Poor research practices are clearly an important problem and warrant a maintained focus on improving education and culture. But if we ignore systemic problems that drive such failings, such as the career imperatives for academics to obtain publishable (i.e., ‘positive’) results, these dysfunctional incentives will continue to shape behaviours in insidious ways.
A related concern is that our proposals would burden honest researchers with ‘red tape’. Actually, declaring interests and submitting documentation through centralised portals should reduce reporting burdens, while the experience from registered reports suggests this format can be readily incorporated into journals’ workflows.
“These are not new ideas”
We couldn’t agree more! This objection seems to reflect academia’s pre-occupation with proprietary innovation and novelty. The problems in research have been discussed for decades. It seems a better bet to focus on implementing well-understood solutions, rather than holding out for the hope that some ingenuous and painless solution to all these problems is just around the corner.
“I don’t want to look foolish by putting my name to a campaign that might not succeed”
Given just how entrenched our problems are, idealism about trying to change things for the better can easily look like naivety, particularly in the cynical world of academia. Many of us in research are by nature contemplative and relish nuance. Calling for specific change can feel unseemly or a kind of activism. I share this discomfort, but also believe that any negligible reputational embarrassment is easily justified by even a remote possibility of helping to improve our deeply flawed research system.
We need your help
If you would like to find out more about the declaration visit www.improvehealthresearch.com. If you think that adopting the measures outlined in the declaration could improve research then please:
- consider signing the declaration
- tell your friends and tweet @ImproveHealthR
- to get more involved, please get in touch
- Or, If you disagree, we would still love to hear from you
Acknowledgments: I would like to thank HealthWatch for their guidance and support and to all those who have contributed and given feedback on the declaration
Conflicting Interests: I am on the steering group for the declaration to Improve Health Research, I receive funding for my PhD research from Cancer Research UK. I am a member of the executive committee of the Fabian Society which is a think tank linked to the UK Labour Party. A full disclosure of interests is available via the University of Leeds website