Now that we have an inkling of how to measure goodness in terms of healthy human life years saved, we can return to the question of how one knows the effectiveness of our actions. If we assume that things like race, religion, and nationality of a person don’t make people’s lives any more or less valuable, and all we care about are numbers of healthy human life-years saved (more specifically, DALYs averted), then we can look at the charities we might or might not contribute to, and see where we can get the most bang for our buck. Just as a wise person would do their homework before making a financial investment, we can choose charities that prevent the most DALYs.
On the one hand, the charity with the most money has the potential to avert the most DALYs. But we can’t assume that all charities are working at the same efficiency. What we want to know is our return on investment, so to speak. That is, we want to know how many DALYs are averted for every dollar we put in. (I’m going to speak in terms of American dollars for this discussion, but the reasoning is the same for any currency. I’m choosing American dollars because I assume most readers can more easily mentally convert their own currency into USD than any other currency.)
Charity’s Dark Ages
In the past, this information was completely unavailable. I’ll call this charity’s Dark Ages because we were really in the dark about what good they were really doing. This is not to say they did no good, we just didn’t know how much. Not only did the giving public not know, but the charities themselves didn’t know, and they didn’t take any steps to find out.
Measuring the good the charities actually do is an embarrassingly new concept. Before the early 1990s, nobody had done it at all.1 This might sound unbelievable, but their reasoning wasn’t completely brain-dead: finding out how effective you are takes resources that could be spent on actually helping people! So they preferred to have faith that what they were doing helped, rather than investing time and money into finding out how much (or indeed, if) it did.
Once, I was talking to a scholar who was (and still is) one of the world’s experts on computer science education. He had a teaching method he advocated and used in his university classroom. I asked him once if he wanted to run a study, where he taught one class in a traditional way (the control group), and another class in the new way. Then he could look at learning outcomes and see how much better the new way was. He said he didn’t want to do that, for ethical reasons. He was so sure that his way was better that he thought it wouldn’t be fair to the students in the control group to teach them in the traditional way.
He had a cold at the time, so I sold him some leeches to help him get over it.
Such is the reasoning behind why charities don’t want to measure what good they’re doing. The big problem with this is that when you do make the effort to measure the amount of good done, it turns out that some charities matter (they are doing some good), other charities unmatter (they’re doing nothing at all), and some charities even antimatter (they’re doing harm).
The measurement of the effectiveness of charities was started by Michael Kremer and Rachel Glennerster around 1993, who worked with a charity to try to improve education in Kenya. If you wanted to improve education for Kenyan kids, what would you do? Think about what your ideas would be before reading on.
Got a list? Whatever’s on that list, these two probably tried it. They tried providing textbooks. But that only helped the students who were already doing well. They tried hiring more teachers to reduce class size. That didn’t make any difference. Over and over, the things they tried turned out to unmatter when they measured the outcomes. I want to emphasize here that these were all interventions that common sense suggests would be really good ways to improve education.2
Okay, but what about the charities that actually antimatter? What are these antimatter charities and what kind of monsters are dedicating their lives to running them? They’re not just charities working toward something you fundamentally disagree with (such as pro- or anti-abortion charities), but ones that are trying to do things that are kind of universally agreed-upon as good.
A pioneering team decided to try to measure whether their own charity was doing any good. The famous Cambridge-Somerville Youth Study looked at a treatment that seemed very promising: they would find children at risk and mentor them. These mentors helped with psychiatric problems, financial problems, health, etc. What was new was that they compared these treated kids with a control group—kids in similar situations who got no treatment. This had never been done before. They looked at forty-three different ways the kids’ lives might have improved over the course of the next thirty years, including health measures, criminal activity, and family issues.
The program did not work. The treatment group was no better than the control group on any measure. The shocking finding was that not only did the program not work, it did harm: for seven of the measures, the treatment group was worse off than the control. That is, the poor, disadvantaged students who got mentored and helped by people ended up worse off. People are still debating exactly why, but there is little doubt that the well-intentioned, hard work done by these people did more harm than good. This is sobering. The peer-mentoring, as it was done, antimattered.3
This is not an isolated incident.
One development group built a fish factory to try to improve the lives of the hunter-gatherers who lived on Lake Turkana. The lake got overfished and the fish stock collapsed.4
“Scared Straight” was another well-intentioned intervention that tried to get kids to have better lives. It involved having incarcerated people come talk to kids at school, telling them about why they should not follow the same path they did. It backfired. The criminal life was romanticized and the kids who were “scared straight” had increased delinquency.
Scared Straight was not based on sound criminological principles. The Adolescent Transitions Program, however, was. It promoted self-monitoring, communication skills, and pro-social goal-setting. Yet even in this program, delinquency was increased three years later, presumably because grouping problematic people together undermines the treatment.5
It’s hard for a charity to know what to do, and it takes a lot of extra work and money to see if whatever they’re doing works. Sometimes, for educational, criminal, or health interventions, these measurements must be done years after the intervention.
I hope these examples have seriously undermined your trust in any charity that doesn’t do evaluation. Doing charity well isn’t easy.
Let’s get back to the Kenyan schools. Kremer and Glennerster finally found something that worked: based on a tip from a friend at the World Bank, they tried something that didn’t seem to have anything to do with education: deworming. It worked really, really well. Not only did it improve education, but it generally helped with the health and economy of the area.6
What I love about Kremer and Glennerster’s story is that this solution, which was very cheap, and had such amazing benefits, isn’t something that most of us would think of when we try to imagine improving education. (Was de-worming, or any health intervention, on your list of ideas of how to help the education of the Kenyan children?) People who study education often don’t learn much about disease, and people who study disease often don’t learn much about education. But in this case all the common-sense interventions unmattered, and preventing disease mattered.7 Do you care more about health or education? In this case, it didn’t matter: helping health was better for both causes.
These new evaluations show that the differences in effectiveness between the best charities and the merely good ones can be huge—the best being sometimes hundreds of times more effective.8
We are living in an exciting new era where we now have the tools and the motivation to do better than using our instincts about how to make this world a better place. When it comes to donating money to charity, the way is pretty clear: choose a charity that, according to evidence, averts the most DALYs per dollar. What charities do that? You can always look at GiveWell.org for their top recommended charities. At the time of this writing, some of the best charities are the Against Malaria Foundation, GiveDirectly, and Evidence Action’s Deworm the World Initiative.
But humans aren’t the only creatures that can have good or bad feelings. Let’s turn to animals, and see what we can do there.