One thing you hope for, with politicians, is that they won’t make the same mistakes over and over again.
Last week we saw that the government has overstated the problems in the NHS by using dodgy figures (to be precise, it used misleading static figures instead of time trends). We also saw that Andrew Lansley’s frequently repeated claim – that his reforms are justified by evidence – is untrue: the evidence doesn’t show that price-based competition improves outcomes (if anything, it makes things worse); and the evidence also doesn’t show that GP consortiums improve outcomes (unless you cherry-pick only the positive findings). It’s OK if your reforms aren’t supported by existing evidence: you just shouldn’t claim that they are.
Now Lansley’s Junior Minister Paul Burstow MP has kindly responded, repeating the exact same mistakes again, only more clumsily. From a Minister, this is frightening, so we should see how he does it.
Burstow’s letter:
In Ben Goldacre’s pursuit of the evidence for NHS modernisation (‘Evidence? What evidence?’, 5 February), he appears to have overlooked the impact assessment we published alongside the health and social care bill, where we present a thorough analysis of the evidence for and against our plans. As he will see, studies show that GP fundholding and practice-based commissioning delivered shorter waits and fewer referrals to hospitals for patients. The evidence on competition demonstrates that when it is well-designed and conducted on the basis of quality (as we are proposing), rather than price, it can drive up quality and efficiency.
We have not sought to understate the achievements of the NHS – but a 2008 study by Martin McKee and Ellen Nolte, citing OECD data, concluded that the UK had one of the worst rates of mortality amenable to healthcare among rich nations. If the NHS was to perform as well as the best-performing countries, thousands of lives could be saved each year. We make no apology for that.
Finally, Goldacre appears to have misunderstood the aims of our plans. We are not advocating reform for the sake of ideology. The changes we are proposing are designed to put patients first, improve health outcomes, empower clinicians so they can design services that meet the needs of their local communities, and put the NHS on a more sustainable footing so it is better able to meet the challenges of the 21st century.
Paul Burstow MP
Minister for Care Services
The government initially claimed that UK heart attack death rates were twice as bad as those in France. This was an overstatement: they are, but following recent interventions the gap is closing so rapidly that on current trends it will have disappeared entirely by 2012. In response, Burstow cites a 2008 paper by McKee and Nolte which he says ‘concluded that the UK had one of the worst rates of mortality amenable to healthcare among rich nations’.
Burstow either misunderstands or misrepresents this very simple and brief paper. It is a study explicitly looking at time trends, not static figures, and it once again finds that comparing 2003 with 1998, the UK still had fairly high rates of avoidable mortality, but these were falling faster than in all but one of the other eighteen industrialised countries they examined. (Meanwhile, in the US, avoidable mortality improved at a disastrously slow pace, although they spent more money).
This is a paper showing the recent success of the NHS, and the fact that we are discussing such a massive improvement in avoidable mortality from Labour’s first term in government is not my being partisan: this is the paper that was cited by the Tory Minister as evidence, bizarrely, of the NHS’s recent failures.
Next, Burstow says I ‘overlooked the impact assessment we published alongside the health and social care bill, where we present a thorough analysis of the evidence for and against our plans … studies show that GP fundholding and practice-based commissioning delivered shorter waits and fewer referrals to hospitals for patients’.
In its section on GP fundholding, this ‘thorough analysis’ ignores the four peer-reviewed academic papers I described last week, which sadly found no evidence of an overall benefit from GP fundholding. It makes a series of five assertions about outcomes, though not one of these is referenced to any single paper or study, anywhere at all.
I contacted the Department of Health, which ferreted out the sources especially for me. It turned out there was just one, a document from the King’s Fund. It’s not a peer-reviewed academic journal article, but the King’s Fund is pretty good, in my view. If you read this document, once again, as with other reviews of the literature, it finds that the results of GP fundholding were mixed: some things got better, some things got worse.
So, the Minister has cherry-picked only the good findings, from only one report, while ignoring the peer-reviewed literature. Most crucially, he cherry-picks findings he likes while explicitly claiming that he is fairly citing the totality of the evidence from a thorough analysis. By using this approach, I can produce good evidence that I have a magical two-headed coin, if I just disregard all the throws where it comes out tails.
Here is what politicians apparently cannot understand, repeatedly, over and again: it’s fine to make policy based on ideology, whim, faith, principles, and all the other things we’re used to. It’s also fine for evidence to be mixed. And it’s absolutely fine if your reforms aren’t supported by existing evidence: just don’t pretend that they are.