Politicians Can Divine Which Policy Works Best by Using Their Special Magic Politician Beam

Guardian, 22 May 2010

We have accidentally elected a coalition. This week all good citizens are poring over the ‘Programme For Government’, and there is much to be pleased with. Labour wasn’t all about unbridled credit and fun public sector spending sprees: they kept all our emails, kept records of the websites we visited, used ‘anti-terrorism’ legislation on people who plainly weren’t terrorists, and so on.

But most interesting are the noises now being made by the coalition on crime and evidence. ‘We will conduct a full review of sentencing policy,’ they say, ‘to ensure that it is effective in deterring crime, protecting the public, punishing offenders and cutting reoffending. In particular, we will ensure that sentencing for drug use helps offenders come off drugs.’

These are grand promises. Compulsory addiction rehabilitation with ‘Drug Testing and Treatment Orders’ was introduced precisely ten years ago – as an alternative to custodial sentences or simple probation – for those who have committed drug-related crimes. Their implementation without adequate analysis is a graphic example of our failure to run simple trials of social policy.

Any judge making a decision on a criminal’s sentence is in the exact same position as a doctor making a decision on a patient’s treatment: both are choosing an intervention for an individual in front of them, with the intention of producing a particular set of positive outcomes (reduced crime, say, and reduced drug use); both get through a large number of individuals in a month; and in many important situations, neither yet knows which of the available interventions works best.

If you randomly assign a fairly large number of criminals, or patients, to one of two interventions, in situations where you don’t know which intervention would be best, and measure how well they’re doing a year or so later, you instantly discover which intervention is best. Add in the cost, and you know which is most cost-effective. The basic principles behind this idea are not new, and were first described in the Old Testament, Daniel 1:12.

Before being rolled out nationally in October 2000, DTTOs were extensively piloted in three cities by the Criminal Policy Research Unit of South Bank University, at considerable cost. What insights did this generate? There was no randomisation, and no ‘control’ group of identical criminals given traditional sentences for comparison. Because of that, the only new knowledge generated by these pilots was the revelation that it is possible to set up a DTTO service and run it in some buildings in some cities.

As it happens, when they did follow up the people who had passed through the service, they hadn’t done particularly well. But this finding wasn’t published until after the service had been rolled out. In any case, because there was no randomised comparison group, we have no idea how these participants would have turned out if they’d been given a traditional custodial sentence anyway, so there’s no sense in giving those results even a moment’s thought.

This is a tragedy, and not just because drug use is estimated – with the usual caveats about estimating more nebulous stuff – to be behind 85 per cent of shoplifting, 80 per cent of domestic burglaries, over half of all robberies, and so on. It is also a tragedy because it speaks to motives that will never go away.

We would need a very brave modern politician to say: ‘Look, I want to introduce a new policy, but I honestly don’t know if it will work’. We might need to be forgiving, ourselves, of someone who said this, and actively encourage them to try out their ideas on half of a group of people. We would need a political class that could react to deferred outcomes, with the results dripping out from new interventions over the course of years, perhaps long after the one initiating politician has moved on. Doing all this would revolutionise social policy, in far wider domains than just criminal sentencing. But for now, politicians – and we must share the blame – get further when they use rhetoric, and absolutes.