This week the Food Standards Agency published two review papers showing that organic food is no better than normal food, in terms of composition, or health benefits. The Soil Association’s response has been swift, and it has been given prominent, blanket right of reply throughout the media. That is testament to the lobbying power of a £2 billion industry, and the cultural values of journalists. I don’t care about organic food, but I am interested in bad arguments. The Soil Association has made three.
Firstly, it says that the important issue with organic food is not the personal health benefit (there is none to be found in this data), but rather the benefit to the environment. This strategy – ‘Don’t talk about that, talk about this’ – is a popular one, but it is cheap: we can talk separately about the environmental issues with organic food, but right now we are talking about the health effects.
Secondly, it says that the health benefits of organic food are related to pesticides, and cannot be measured by the evidence that has been identified and summarised in the FSA paper. This, once more, is gamesmanship. There was no evidence of health benefits to individuals. Possibly the Soil Association is proposing that there are health benefits which somehow cannot be measured: it is hard to disentangle the health benefits of eating organic food from other beneficial features of people’s lives, for example, when you measure their health in a thirty-year study. In this case it is expressing a position of faith, not evidence. Or it is proposing that there are health benefits which could be measured, but have not been measured yet. In that case, again, this is faith rather than evidence, but it could start recruiting researchers now, using some small portion of its industry’s £2 billion revenue to investigate these beliefs with fair tests.
Lastly, like many vitamin-pill peddlers and pharmaceutical companies before it, the Soil Association seeks to undermine the public’s understanding of what a ‘systematic review’ is. It says that the report has deliberately excluded evidence to produce the answer that organic food is no better. The accusation is one of cherry-picking, and it is hard to see how it can be valid in the kind of studies that have been published here. These are ‘systematic reviews’: before you begin collecting papers to include, you specify how you will search for evidence, what databases you will use, what types of studies you will use, how you will grade the quality of the evidence (to see if it was a ‘fair test’), and so on.
What does the Soil Association think these systematic reviews have ignored? As an example, from its press release, the industry body is ‘disappointed that the FSA failed to include the results of a major European Union-funded study involving thirty-one research and university institutes and the publication, so far, of more than one hundred scientific papers, at a cost of 18 million Euros, which ended in April this year’. It gave the link to www.qlif.org.
On this website, you will find the QLIF list of 120 papers. Almost all are irrelevant. The first fourteen are on ‘consumer expectations and attitudes’, which are correctly not included in a systematic review of the evidence on food composition and health. Then there are twenty-two on ‘effects of production methods’. Here you might expect to find more relevant research, but no.
The first paper (‘The effect of medium term feeding with organic, low input and conventional diet on selected immune parameters in rat’), while interesting, will plainly not be relevant to a systematic review on nutrient content. The same is true of the next paper, ‘Salmonella Infection Level in Danish Indoor and Outdoor Pig Production Systems measured by Antibodies in Meat Juice and Faecal Shedding on-farm and at Slaughter’: you might love its results, you might hate them; either way, it doesn’t matter. This paper is simply not relevant to a review on nutrient content.
What’s more, the overwhelming majority of these studies are unpublished conference papers, and some are just brief descriptions of the fact that somebody made an oral presentation at a meeting. The systematic review correctly looked only at good-quality data published in peer-reviewed academic journals, and with good reason: we know that conference papers are unreliable sources of information, that they often change between conference and publication, and that they are often never published at all.
As so often, this is about transparency, which is ultimately the only source of authority in science: we want the methods and results of scientific research to be formally presented, and accessible by all, so that we can see what was done, and what they found. If a government report on anything relies substantially on unpublished and inaccessible research, then we are correctly concerned. In fact, just two weeks ago this column discussed how the key piece of evidence presented by the Home Office to justify retaining DNA from innocent people who have only ever been arrested and then released was an incompetently presented piece of unpublished, incomplete research.
Systematic reviews give you transparency, because from the very outset everyone is honest and open about what kinds of studies they will and won’t include. When organisations like the Soil Association greet the publication of a systematic review with accusations of cherry-picking, they’re not just wrong, they’re undermining the public’s understanding of one of the most important and simple new ideas from the past three decades of science.
That annoys me, so let me share a prejudice. Ultimately, when people talk about the health benefits of organic food, they’re not really talking about the health benefits of organic food. Wealthy society’s big love for organic farming is about something very different: a bundle of legitimate concerns about unchecked capitalism in our food supply, battery farming, corruptible regulators, and reckless destruction of the environment, where the producers’ costs do not reflect the true full costs of their activities to society, to name just a few. Every one of these problems deserves our full, individual attention.
But magic spells will not work. We cannot eradicate deceit from the pharmaceutical industry by buying homeopathic sugar pills; and we cannot solve the problems of unchecked capitalism in industrial food production by giving money to the £2 billion organic food industry in exchange for the occasional posh carrot.