Trustworthy communication of risk and evidence: the battle against naughty numbers in the news
A talk by David Spiegelhalter, Chair of the Winton Centre for Risk and Evidence Communication, Centre for Mathematical Sciences, University of Cambridge; Non-Executive Director, UK Statistics Authority; President, Royal Statistical Society 2017-2018. Professor Spiegelhalter spoke at the Medical Society of London on 27th July 2021. He is joint winner of the 2021 HealthWatch Award. The following is an edited transcript. The full recording can be seen on the HealthWatch YouTube channel.
Tonight I'm not going to talk about the construction of the evidence but about the messaging - the storytelling around the numbers. The American statistician Nate Silva says the numbers have no way of speaking for themselves, we speak for them. That means the numbers have to be embedded in the context and there has to be a narrative around them in order to see what their value is.
I'm retired. I was going to have a nice gentle retirement and then Covid came along and suddenly everyone wants to talk to a statistician. But people will attack you. I was on Andrew Marr last week, and afterwards someone tweeted to him, “Well done, you've just put people off getting vaccinated by listening to that idiot David Spiegelhalter. You never question his findings which are completely wrong.”
I think that the statisticians and the public health organizations have done a great job. From my perspective there's a huge demand from the media for experts. There's been brilliant work by the UK Science Media Centre, an organization that puts scientists and terms in in touch with the journalists. Most journalists, particularly the specialist journalists, have acted very well. But it's tricky to remain non-aligned. The media love to split and blame and get arguments going. When I am asked, “why do you think this is happening?” and I reply, “I have no idea you'll have to wait and see” they won’t use that quote.
Putting heads above the parapet
I've written an article with Kevin McConway to help professional statisticians deal with the media. Some lessons: sound human; make friends with journalists; beware of “just a chat” - never have just a chat with a journalist, they're recording every word and they're going to use that one line where you slipped up and went all casual. You’ve just got to pick yourself up try to ignore the criticism and get going again.
We’re encouraging people from the Royal Statistical Society to get out there and put their heads above the parapet. Because I think the public deserve and need what I call non-aligned scientists.
The media are hopeless with denominators. Here’s one: “Only 0.005 percent of coronavirus deaths have been children.” That's completely wrong! Only 0.005 percent of children with the virus have died. So the denominator is not coronavirus deaths, it's children with coronavirus, in other words they've got a very low death rate.
That was the fault of the sub editor – the people who write the headlines. The journalist didn't write the headlines, he wrote a perfectly good article but someone else stuck a completely inappropriate headline on it.
Social media
I'm going to talk on the whole about mainstream media, but with a brief mention of some of the awful stuff on social media. One tweet said that a new study suggested that more than five million Britons have had the coronavirus (correct so far); and went on to say, given that 50,000 people had at the time died from it (correct) this means it has an infection fatality rate of less than 0.1 percent – wrong! 50 000 out of 5 million is one percent, not 0.1%. This person had been arguing for six months that Covid was no more dangerous than flu and he thought he'd found the statistics to prove that, buy by doing so he showed he was wrong the whole time.
Diehard skeptics of that variety have actually had very limited mainstream media coverage. They get lots of retweets among that particular coterie but actually in terms of the mainstream coverage it's very limited. Which is very reassuring.
I'd like to go on to the idea of pundits speaking outside their expertise. Lord Sumption, ex-Supreme Court judge was interviewed on the BBC Radio 4 Today programme on July 20th where he made three statements. The virus has not killed over 100,000 people; the people who died would probably have died within a year after anyway; and the number of people who have died without a sufficiently serious co-morbidity to appear on the death certificate is very small - a matter of hundreds and not thousands. All of these are totally wrong! Why wasn't he challenged at the time?
Facts checked
In the event, Full Fact, the fact-checking charity, destroyed the inaccuracies in that interview within hours. By that day 124,000 people had died with Covid as the underlying cause of death. People dying of Covid had lost about a decade of life on average.
Now, an area of dispute is around the fact that 90% of people who die with Covid do have underlying health conditions. In reality, Covid picks on any vulnerability you have and exaggerates it, it essentially takes the normal risk anyone has and pumps it up so if you have any vulnerability for any reason it'll hit it really hard. It’s a real bully.
But while 90% of people do have something else along with Covid on their death certificate, the other 10% haven't. So there were not a few hundred, but 15,000 death certificates that gave nothing else but Covid as the cause of death, just in England and Wales.
The fact that misinformation is appearing in that mainstream media is very bad, but fortunately there hasn't been that much of it.
Covid in context
A graph that came out this morning from Office for National Statistics shows how many deaths were registered over the last five years, which lets us see Covid in the context of the normal pattern. It shows that with the first peak of Covid there were also quite a lot of excess non-Covid deaths - almost certainly many of these did have Covid but it didn't go on the death certificate because they were in care homes, or weren't seen by the doctor.
Then in the second wave there is another spike of Covid deaths but here, the number of non-Covid deaths is running below average. The crucial question is, are the missing non-Covid deaths we saw in the second wave, just some of the people who died in the first wave but who would have died naturally a year later if they hadn’t?
There is something called mortality displacement. There are certainly fewer non-Covid deaths in the second wave because some of them died a year earlier in the first wave, but that doesn't fully explain what's going on in this graph. One of the reasons is: there's been no flu.
If you look at Covid hospital admissions last winter alongside flu hospital admissions, flu is a flat red line – there were practically none. It means we quite possibly could be hit badly by flu in this winter coming up. So the lockdown could have prevented 10,000 to 15,000 deaths from flu over the winter. I’m not saying we should always have a lockdown every winter, but it does seem to have saved lives from flu.
Let's talk about trust
Let’s finish off by talking about trust. Baroness Onora O’Neill did a fantastic series of Reith Lectures some years ago on the subject of trust. One of her great ideas is that organizations should not be trying to be trusted, instead there's a duty to demonstrate trustworthiness. This is a brilliant switch of the responsibility! For people to trust you, you’ve first got to demonstrate you can be trusted. She has had a huge impact. The Code of Practice for Statistics has trustworthiness as its first pillar.
In our Winton Centre we've been trying to codify what trustworthiness means in an era like this where you're trying to communicate complex evidence. We've got it down to five things:
- Inform people in order to empower them to make informed decisions rather than manipulate them to do what you want them to do. And if you do want to manipulate them, be honest with yourself that you are just advertising. Just own up.
- Balance, but not false balance - show the positives and the negatives but don't pretend they're equal.
- Disclose uncertainties.
- State the quality of the evidence. Be upfront if the evidence isn't very good if your statistics are unreliable.
- Pre-bunk misinformation. If people are spreading misinformation get in there hard and counter it and tell it to people even before they hear it. There is good evidence this can be effective. “Do you know there's people going around saying that vaccines make you infertile?” Get in there first and counter it.
Back in April, Jonathan Van Tam, deputy Chief Medical Officer for England, a trusted communicator, used our slides to tell people under 30 that they shouldn't get the Astra Zeneca vaccine because of the risk of blood clots. It's a complex argument, and the balance of risk changes depending on the likelihood of exposure to the virus. It may have changed because of how common the virus now is. They explained that the benefits of the vaccine for people my age in terms of preventing intensive care admissions, is huge. But when exposure levels are low, the benefits of getting the vaccine are much less for 20 to 29 year olds because that age group is very rarely going to end up in intensive care for Covid. For the Astra Zeneca vaccine, these rare clots are more common in younger people. So you've got these counteracting risks and benefits which change with age, and at some point the balance flips over and so they initially set the limit for recommending the vaccine at age 30 and above.
It would be quite reasonable now to change that flip point because exposure risk to Covid at the moment for young people is high. The risks of the vaccine stay the same but the risk of being exposed to the virus goes up. But that was our attempt to show balance. It went down very well.
Pre-debunking doesn't always go well
An example that didn't go down so well, was an attempt to pre-debunk misinformation. The official data for deaths from the delta variant up to June 21st this year, showed that among the over-50s there'd been 38 deaths among unvaccinated people, 18 among people with one dose of vaccine, but 50 among the people who had had two doses of the vaccine. So we wrote an article in The Guardian entitled “Why most people who now die with Covid in England have had a vaccination”, to pre-empt the inevitable misunderstanding that if the vaccinated are dying at a higher rate than the unvaccinated, vaccines don't work.
Really it’s because so many people have been vaccinated. The actual risk of dying among the vaccinated is roughly one-twentieth of the risk of dying for people who are not vaccinated, it’s the same for each age group. If everybody had been vaccinated then anyone who died would have been vaccinated so it's not an indication of the failure of the vaccine, it's an indication of the successful rollout of the vaccine.
We tried to explain, and I think we failed because we got picked up by vaccine sceptics, who only used the top title. I still believe it's right to pre-empt misinformation. Whatever you say will be manipulated but it's best to get it out there honestly rather than be on the back foot and reacting.
Scientists should rise above polarized policy debates
This article came out yesterday in The Conversation, “COVID: media must rise above pitting scientists against each other – dealing with the pandemic requires nuance.” I disagree. Why blame the media? Shouldn't scientists themselves rise above polarized policy debates? You should expect to be argued with don't expect to get away with it just because you’re a professor. If you're going to start telling people what to do you've got to be prepared to take the hit. Mainstream media have in general done a reasonable job at trying to fend off misinformation. Scientists have tried hard to explain but some have become advocates for policy and that's fine if you're representing a named organization, or if you're speaking as an individual, but this is not part of being a professional scientist – scientists should be speaking on behalf of science.
I feel very strongly that this polarization with scientists taking policy viewpoints has been one of the worst things of this pandemic and, if anything, more harmful than the misinformation campaigns that have been around.
Be careful speaking to the media. I did an interview with the Today programme before Christmas, and I was talking about precautions and how I can't sing in my choir anymore, and then I said “Maybe they should ban arguments over the Christmas table because shouting at each other in a small unventilated space is one of the worst things you can do.” So this was a joke. In the Daily Express next day we saw: “Christmas warning: families could be banned from arguing to prevent Covid spread, according to a leading British statistician.”