NEWS: Peer review and the advancement of scienceDr Terence Kealey, Vice Chancellor of the University of Buckingham, has a highly eccentric view of scientists, and of their fellow-scientists who review the research papers submitted for publication in scientific medical journals. In an opinion piece in the Daily Telegraph (19th February 2008) he claims that peer review is supposed to combat fraud, but can just as easily hold back radical discoveries. I disagree.
Do not look in general journals (such as the New England Journal of Medicine, Lancet, BMJ or JAMA) for “radical discoveries”: that is the field in which specialist journals flourish. The editor will (or should) require authors of the paper, and also the reviewers, to declare any conflict of interest. The methods the authors used must be described so clearly that a critical reader can replicate the study. This is the type of journal where the claim of a “radical discovery” is most likely to be made, and where it is most likely to be demolished. Editors do not reject papers because they are contrary to received wisdom: quite the reverse! But if a study, although technically apparently sound, reaches an amazing result the editor will publish the paper, but may add an editorial highlighting the need for replication of the study.
My experience of editing and refereeing medical scientific papers is incompatible with Dr Kealey’s assertions for five reasons.
First, the purpose of peer review is not primarily to combat fraud, and it is not very successful when it tries to do so. Fraudsters are usually unmasked by critics who show that the study cannot be replicated. The reason might be fraud, or (more probably) bad technique or wrong calculation of the data, but for whichever reason it damages the reputation of the researcher, who will have to be extra careful to be correct in future.Second, reviewers are not unduly impressed by a prestigious name among the authors, nor do editors routinely reject papers by an “unknown”. Some journals send out anonymous copy for review to avoid this bias, but this proves counter-productive: the reviewer often wastes time trying to identify the authors rather than concentrating on the quality of the text.
Third: “The myth is that science is the noble search for truth. The reality is that scientists are selfish.” Does that mean more selfish than the Olympic athlete determined to win? Or the vice-chancellor desperate for better funding for his university? I do not think so. I cannot think of any employment in which rewards are so closely related to ability, honesty and hard work than that of a research scientist.
Fourth: “Sometimes, trusting what scientists tell us can be difficult”. This opening statement is followed by examples of difficult-to-trust advice—such as to use/avoid artificial sweeteners, or to drink/avoid coffee. I need to be sure that such sentiments were being expressed by a scientist. Of course the media tell us that the “latest research” reveals something, which is made even more exciting if the next day they can report that the reverse is true. What scientists (by my definition) say is, “This is the experiment we did, and exactly how we did it. The results we obtained seem to imply that artificial sweeteners/coffee is good/bad for you.” I have no difficulty believing this kind of statement unless, or until, a greater weight of experimental evidence points in a different direction.
Fifth: “But peer review carries dangers. First it allows dunderheads to block unexpected ideas … such as Barbara McClintock who won the Nobel Prize in 1983 …”. How many other examples are there of dunderhead activity? And anyway, was Barbara really so badly treated?
Barbara McClintock was born in 1902 and died in 1992. Cornell gave her a PhD in botany in 1927, and by 1931 she had published three substantial papers in distinguished journals (two of them with herself as sole author) Genetics and PNAS. Most of us would think that was quite good progress for a 29 year-old female in 1931. So the dunderheads seem on this occasion to have recognised a good “unknown” scientist rather quickly.
Can Dr Kealey find reviewers behaving as obstructive dunderheads these days? Or is he just being “controversial” to promote sales of his new book?
John Garrow
Emeritus Professor of Human Nutrition
University of London
Reference
1. Kealey T. Peer review: the myth of the noble scientist. Daily Telegraph 19th February 2008. View on: http://www.telegraph.co.uk/earth/main.jhtml?xml=/earth/2008/02/ 19/scipeers119.xmlMore on this in the members' forum - click on the forum button on the left
Top of page
NEWS: Screening “halves breast cancer deaths”: claim questioned
Results of a study claiming that routine breast screening halved death rates in East Anglia have been questioned by top breast cancer surgeon Michael Baum.
The Eastern Daily Press reported on 10th January that, according to a Cancer Research UK study published two days earlier in the British Journal of Cancer, screening in East Anglia has reduced the number of deaths from breast cancer there by 48%. But Michael Baum has told HealthWatch, “The paper in the British Journal of Cancer is seriously flawed.” …
References
- Marsh, L. Breast screening cutting deaths by half. Eastern Daily Press, 10th January 2008. A case-control study of the impact of the East Anglian breast screening programme on breast cancer mortality.
- Allgood PC, Warwick J, Warren RML, Day NE & Duffy SW. Brit J Cancer 2008; 98: 206–209.
Top of page
NEWS IN BRIEF
The online forum for HealthWatch members is now live and can be accessed on http://healthwatch.informe.com/forum/ (or by clicking on the “members’ forum” button on the left of the screen). There are notes there to help you to register for the forum, read what has already been posted, reply to what has been posted or create a new topic. “I hope that we can use the forum to stimulate debate and discussion between HealthWatch members,” said David Bender, HealthWatch chairman, who will personally monitor the forum, and ensure that any inappropriate material will be removed. Offending authors may be banned from the forum.
Pressure is growing on the government to withdraw NHS funding from homeopathy. An e-petition on the 10 Downing Street website asks the Prime Minister to immediately ban NHS funding of homeopathy and redirect the resources toward proven medicine. The petition is open until 21st August 2008. See http://petitions.pm.gov.uk/anti-homeopathy/
Health tests for the “worried well” can actually be harmful, warn scientists and GP’s in a new booklet. Despite the explosion in marketing of scans and check-ups to healthy people, most tests weren’t designed for well people and are not researched or adequately regulated. Making Sense of Testing is published by Sense About Science with the Association of Clinical Biochemistry, the PHG Foundation and the Royal College of Pathologists. GP contributor Dr Andrew Green advises, “If you don’t have symptoms, then very few tests are worthwhile, and those that are can be had through your doctor.” The 16-page booklet is free and can be ordered or downloaded online at: http://www.senseaboutscience. org.uk/index.php/site/project/232/
Members of HealthWatch are reminded that the subscription rates increased this year and a note may be enclosed with this newsletter asking that members who pay by cheque should kindly pay before June in order to maintain their subscription, and members who pay by standing order to amend it if they have not already done so. The new rates appear with the membership details in the box on page 8 of this issue.
Scientist, writer and HealthWatch committee member Les Rose has started his own online blog on critical thinking, science and scepticism. It kicked off on 2nd March with his commentary on an extraordinary example of irrational argument and misinformation: a speech by Ivan Lewis (Parliamentary Under-Secretary, Department of Health) given in parliament on 19th February. Bookmark it: http://majikthyse.wordpress.com/
Top of page
News:Cancer position paper now onlineCancer is becoming more common, but death rates are falling, and the reasons are explained in a detailed and authoritative new briefing paper specially prepared for HealthWatch. This latest position paper, Cancer, is now available in pdf format to view or download from the HealthWatch website…
View or download the paper in pdf format from here.Top of page
Festschrift honours Michael Baum
What is a festschrift? Answer: a very rare event that demonstrates an exceptional measure of affection and respect for a distinguished colleague.
My invitation to the Festschrift organised by UCL to honour Mike Baum was the first time I had even come across the term…
Diana Brahams
Barrister
Old Square Chambers, Gray’s Inn, LondonTop of page
New face on the committee
A highly respected clinician and researcher has joined the HealthWatch Committee. Susan Bewley is a consultant obstetrician in maternal-fetal medicine at Guy’s & St Thomas’ NHS Foundation Trust...
Top of page
The evidence for homeopathyThere is evidence associated with homeopathy, but how reliable is it? David Bender has made a detailed and objective examination of the research review which is held up by the Society of Homeopaths as support for claims of efficacy. He found that conclusions reported by the SoH are not always supported by the research from which they are drawn.
In a radio broadcast on December 1st 2007, Jayne Thomas of the Society of Homeopaths mentioned a review of papers in peer reviewed journals that support the efficacy of homeopathy. She has provided HealthWatch with a copy of the review, produced by the European Network for Homeopathy Researchers (ENHR) in November 2006. This document includes a number of individual papers and, more usefully, a number of meta-analyses and systematic reviews of published papers. For each review paper cited in the ENHR document, I have placed the statement from the document in a shadowed box, followed by notes of what the original paper said. I have kept to the order in which the ENHR document cites the reviews…
Summary
Some meta-analyses do suggest evidence of beneficial effects of homeopathic treatments, but the ENHR summary is overly optimistic in its interpretation of the results. More importantly, the original reviews note the twin problems of poor study quality and publication bias (the fact that negative results are less exciting to offer for publication, and less likely to be accepted for publication). It is clear that the high quality studies, and especially those included in the more recent meta-analyses (e.g. reference 7) do not show any effect of homeopathy above placebo.
David A Bender
Senior Lecturer in Biochemistry
University College LondonTop of page
CAM therapies: Sharks and cancer: a myth is bornDr Judah Folkman, who died in January 2008, introduced the concept of angiogenesis, the idea that tumours secrete a substance that generates their blood supply. He went on to prove it. And, in the course of research, his team inadvertently triggered the lunatic idea that shark cartilage prevents cancer. This is how it happened…
Caroline Richmond
Medical JournalistFurther information
Dr Folkman’s War: angiogenesis and the struggle to defeat cancer by Robert Cooke was published by Random House, New York and Toronto, in 2001.
Caroline Richmond’s obituary of Folkman was published in the Independent on 4 February 2008.
Top of page
CAM regulation: a new forum thread
Placebo labels for CAM products—could it work? A HealthWatch member e-mailed with this interesting suggestion for how to regulate and label complementary and alternative treatments.
We will be posting this item on the new HealthWatch online forum along with the response below from committee member Les Rose, and we hope that members will join the discussion.
The suggestion:“I believe that regulation based on the ‘Evidence basis’ is impossible with CAM treatments. It would be better to enforce clear labeling and promotion of all products and therapies perceived to be in the CAM area (including ‘Food Supplements’) that have not met the modern efficacy criteria with the bold writing, ‘This product/therapy is a PLACEBO and has no proven health benefit’. Any unsafe product could be banned (if a product contained Comfrey for instance) with ease. All CAM products and therapies would be classified at a stroke as a single group ‘perceived health enhancing treatments unlicensed as medicines or treatments’…”
We would be interested to hear from other HealthWatch members, via the Forum on http://healthwatch.informe.com/forum
Top of page
Research: Stephen E Straus (1947-2007): an obituary and some second thoughts on research prioritiesStephen Straus was a distinguished virologist-immunologist when in 1999 he took office as director of the American National Center for Complementary and Alternative Medicine (NCCAM), which falls under the US National Institutes of Health in Bethesda.
Straus died , aged 60, on 14th May 2007 having suffered from a brain tumor. The NCCAM—the successor of the Office of Alternative Medicine (established 1992)—was formed in October 1998 and was until Straus’ arrival dominated by alternative healers, such as the homeopath W Jonas. Its budget for research was in 1999 already $50 million. Straus was expected to transform the NCCAM into a truly scientific institute…
Cees Renckens MD PhD
Consultant Gynaecologist
Hoorn, The Netherlands
Chairman of the Dutch Society against QuackeryTop of page
Letter to the editorOn Tallis’ difficult moral questions
Gillian Tindall, author, of London, writes:
Dear Editor,As someone who, at one time, used to lend journalistic skills to the Journal of the RSM by doing the occasional seminar report for them, I am a great admirer of HealthWatch. I am also an admirer of Professor Raymond Tallis, and am essentially in complete sympathy with his views on anecdotal ‘evidence’ and the false assumptions that tend to follow from it. (Anecdotes, Data and the Curse of the Media Case Study, published in HealthWatch Newsletter, issue 68, January 2008).
I would, however, respectfully point out that in broadening out his argument to encompass other examples besides the MMR one, he seems to be straying into other and more challengingly complex issues. When he says that NICE decided that “certain drugs would not be cost-effective in the treatment of early Alzheimer’s disease”, the implication is that this decision was based not solely on the drugs’ effectiveness (or lack of it) but also on an ultimate concept of value for money. Lurking behind this there would appear to be the whole vexed and largely taboo question of how much any society, however highly developed, can afford to expend on treating any one individual.
Professor Tallis then moves on to the still more contentious subject of benefit to the individual versus benefit to society, with the clear suggestion that these two aims may be in conflict: “even a treatment that has an adverse effect in some recipients may overall have a beneficial effect on the population.” Well, yes, perhaps—but that would be a highly flawed moral argument with which to recommend such a treatment for general use, and it is one to which many individuals would be understandably resistant.
Let us, by all means, discuss these and other can-of-worms subjects which, in standard medical journalism, are almost universally ignored. But let us not suppose that a better understanding of the nature of clinical trials and a familiarity with elementary logic are going to solve the problems thereby raised, for they will not. On the contrary, greater clarity of thought tends rather to reveal the extent and difficulty of underlying moral questions.
Yours sincerely
Gillian TindallClick here to see Prof Tallis' reply
Top of page
Correction and apology
Before our 2007 HealthWatch Awardwinner, Professor Raymond Tallis, pauses to reflect on the moral questions raised in the letter above, the editor of the HealthWatch Newsletter would like to offer her sincere apologies for the inexplicable error in producing the December issue that resulted in his excellent report of his award acceptance presentation mistakenly being signed off as having been by Professor Richard Tallis.
Mandy Payne
Editor, HealthWatch Newsletter
