8 We Need More Than Control

I first met Tom in February 2019, about a year after I moved from Cambridge to New York. He had emailed me to discuss a business idea related to psychological targeting. Would I be willing to spare an hour and meet him for lunch near the Columbia campus?

Requests from strangers like Tom aren’t unusual. I am often contacted by companies that want to improve the effectiveness of their product recommendations, marketing campaigns, or customer communication. Or by people aiming to establish their own business in that space.

While many of the people I meet genuinely care about improving their customers’ experience, it’s an open secret that—ultimately—their goal is to make money. Happy customers are loyal customers, after all. And loyal customers are profitable.

A quick search on LinkedIn confirmed my suspicion. Tom was working in investment banking. I immediately dumped him into my mental “Wall Street guy” box and adjusted my expectations accordingly.

But I was wrong. As we sipped coffee, he told me he was planning to leave the world of investment banking. And he was committed to using some of the money he made to make a positive dent on the world.

He could have done that in a million different ways. But he decided to tackle a problem I had been thinking about for a very long time: How do we share some of the value generated by personal data with the people who produce it?

If data is the new oil, can we create an economy that benefits not only those who know how to refine it but everybody?

ApplyMagicSauce

Tom and I spent the next three hours brainstorming what a marketplace like this could look like. The idea itself seemed promising. Shift ownership of data to the individual and facilitate an exchange that creates value for both sides. But we quickly identified a common concern: How would people know how much their data is worth?

Think about it: If all your personal data belonged to you, how much would I have to pay you to get access to your entire Facebook profile? Your GPS records? Your credit card histories? You probably have no idea and will likely aim too low.

It’s extremely difficult to understand the value of the digital footprints you generate if you think of them in isolation and don’t have any insights into how they could or will be used. Before I started researching psychological targeting, it would have never occurred to me that my banking app could use the location records it collects to predict my mental health and personality (or more likely to sell this data on to someone else to do this). My location data didn’t strike me as particularly valuable.

But what if we could show people the real, potential value of their data by giving them access to the same insights that companies like Facebook, Google, and X/Twitter use to turn them into cash cows?

Back at the Cambridge Psychometrics Center, we had tried to do this by building an open-access tool that allowed people to submit their Facebook, X/Twitter, or LinkedIn data and receive predictions of their psychological traits. We called it ApplyMagicSauce and put it on the internet free of charge (it’s still there if you want to check it out: applymagicsauce.com).

Tom and I started wondering if we could expand on the idea. We were intrigued by the question whether offering such a self-insights tool could help people appreciate the value of their personal data. Together with Tom’s team and two of my colleagues, Adrian Ward and Martin Abel, we set out to study the question.

As an academic used to limited research budgets, I probably would have patched together a small lab experiment using ApplyMagicSauce. I would have invited a few Columbia students to the lab, showed them the insights we could glean about their psychology, and asked them a few questions about how much they thought their personal data was worth. But Tom had different plans. As a former Wall Street guy, he wasn’t going to play small. He wanted to run the study properly, with real people and a real product.

It took over a year of intense work to build an application that was able to connect to users’ Facebook and Google accounts and generate a broad variety of insights we believed would be of interest to people. It showed them their basic sociodemographic profile—name, address, age, gender—and extracted their Google search histories, Facebook likes, and status updates.

But what Tom’s team had built wasn’t just an app. It was the perfect experimental playground for us to test our hypotheses. Instead of having just one fixed user experience, the app allowed us to compare the impact of different types of “packaging.” How would people react if we only showed them the raw data, the way that Facebook and Google are now mandated to do for their users upon request?

You see a small snippet of what this looks like in figure 8-1. Not exactly user friendly, is it? If anything, showing your data this way might make you value it less, not more. I mean, why would anyone pay for this junk?

But what if we helped people sort through the clutter by presenting the information in a more accessible form? A simple bullet-point list? Or perhaps a more elaborate data story? After all, humans are known to construct their life narratives and identities in the form of stories. Maybe people needed to see their own data story to be able to relate to it.

FIGURE 8-1

Raw data from Google and Facebook

We ended up testing five different insight variations, ranging from the raw data I showed you in figure 8-1 all the way to an elaborate data story. The latter not only included the information Facebook and Google had captured for each user (e.g., all search queries) but also the types of inferences that could be made based on these data points (e.g., credit scores, political ideology, emotions) and potential use cases (e.g., targeted advertising, personalized pricing of loans).

The final decision we had to make before running the study was how to capture the value that people assigned to their personal data. It’s easy to say that you would want at least $50 for your data if the question is purely hypothetical. That’s the angel on your shoulder talking, the one with the good intentions (“Richard, we need to protect our privacy”).

But what if I offered you and a thousand other people actual money for your data? What if I told you that the fifty people who were willing to accept the least amount would receive the extra cash on the spot in exchange for their data. If you’re interested in making some extra cash, you would likely become a bit less ambitious in your demands. At the very least, you would have to think carefully about how much your data is actually worth to you. It’s the angel in conversation with the devil (“Richard, remember that we need to protect our privacy” versus “Forget about privacy, Richard, we want the money”).

That’s exactly what we did in the experiment we ran. We first offered people a deal to sell their data on the spot for $10. After that, we asked them for the smallest amount they would be willing to accept in competitive future bids that could only accommodate a limited number of people. This setup allowed us to gauge people’s real preferences. The choices they made weren’t purely hypothetical. There was real money on the line.

A Dead End

After months of designing the experiment and infrastructure, we were ready to launch. We recruited over 1,500 people online and randomly assigned them to one of the five different insight versions we had (plus a control group of people who didn’t see any insights before being asked to sell their data). As with the other big field studies I had run with Hilton and the beauty retailer, I was nervous. We had all invested an enormous amount of time and money in this experiment. I had written the code to analyze the data weeks in advance. I wanted to be ready and be able to see the results immediately.

When I finally got the data from Tom’s team, I sat down, opened my code, took a deep breath, and hit “Run.” A few seconds later, I could see the results pop up on my screen.

Nothing.

Although 75 percent of our participants said they were concerned or very concerned about data privacy, most of them sold their data anyway (85 percent on average). Worse, there were no differences across the various insight conditions we had worked so hard on. It didn’t matter if someone saw the raw data in junk format, the bullet points, or the full-on data story. They were all equally likely to sell their data and asked for roughly the same amount of money in future iterations. Not only that, people who saw their data in one form or another were just as likely to sell their data as the people in the control group who were asked to sell their data without seeing any of the insights.

To say I was disappointed would be an understatement. A massive understatement. It’s the kind of outcome you dread as a scientist after having spent hundreds of hours and thousands of dollars on an experiment. It felt like an utter failure. I informed the entire team about the results and buried my head in the sand for a few days.

But then something in my thinking changed. I guess that’s what happens when you see the world from a different perspective (upside down, engulfed in darkness, that is). The experiment had failed to produce the results we were hoping to see. But it had still taught us an important lesson (sometimes, no effect is indeed a finding, even if we don’t like it).

Pulling back the curtain and educating people about the insights third parties could derive from their data wasn’t enough to change their behavior. We had equipped people’s angels with better tactical knowledge and weapons. And yet, they stood no chance against the devil’s immediate cash reward.

Toward a Better Solution

This simple insight turned out to be a real epiphany that made me reconsider one of the main solutions I had previously advocated for.

I had been a strong supporter of new data protection regulations like the General Data Protection Regulations in Europe or the California Consumer Protection Act in the state of California. Both regulations are aimed at empowering consumers by mandating high levels of transparency and control.

Empowering consumers has a lot of appeal and in many ways seems like the ideal solution. It’s not only a morally defensible solution, but also allows us to make the most of our personal data. We all feel differently about disclosing personal information and how much we value the services that rely on our data. You might be comfortable sharing your GPS location to get relevant weather updates. I might not. Instead of imposing crippling one-size-fits-no-one regulations, let us decide. At the end of the day, it’s our data, so why shouldn’t we be the ones choosing when and how we want to share it?

Don’t get me wrong. Transparency and control are critical for helping us navigate the tension between the potential harms and benefits of sharing our personal data. And I certainly support the intention behind these new regulations and the fundamental values they represent. But as our failed experiment with Tom suggested, transparency and control alone are often insufficient to help people make the right decisions.

The more I thought about this, the more disheartened I felt. It struck me that in the current data ecosystem, control is far less of a right than it is a responsibility—one that most of us are not equipped to take on. What we get is control without mastery.

Left to our own devices, most of us fail rather spectacularly when it comes to making informed privacy decisions. Our good intentions rarely translate into behavior—a well-established phenomenon known as the privacy paradox. Most of us say we care about our privacy but do little to protect it (just like the people in our experiment). Think you’re the exception? Let me ask you this: When was the last time you updated your privacy settings or carefully read through the terms and conditions before installing an app on your phone? I honestly don’t remember when I did.

But even if we got more people to pay attention to privacy policies, there’s no guarantee they would arrive at the right conclusions. In a survey administered by the legal scholar Chris Hoofnagle at the University of California, Berkeley, 62 percent of respondents erroneously believed that companies posting privacy policies on their website implied that they could not share their personal data with third parties.¹ Sadly, there’s often no incentive for companies to change any of that. The less you care and comprehend, the better.

But why do we have such a hard time managing our personal data responsibly? According to the psychologist Azim Shariff, the answer is quite simple.² Our brains simply haven’t evolved to solve today’s privacy puzzles.

Technology moves at light speed. Evolution doesn’t. The privacy challenges we face today are an entirely new species.³ They have little resemblance to those my grandmother faced even eighty years ago, let alone those our ancestors encountered a century or two back. New technologies have radically altered the privacy landscape over the past few decades. Yet, our brain’s cognitive abilities are essentially the same as two thousand years ago.

In chapter 7, I discussed two common fallacies. Many of us substitute the question whether we care about our privacy with the easier—but different—question whether sharing our personal data is worth it (typically without understanding the potential costs of doing so). And we erroneously conclude that just because we feel like we have nothing to hide or worry about, this makes sharing our data safe. That might be true today, but it might not be true tomorrow.

Add to this a general lack of digital literacy a heap of uncertainty around what companies might be doing with your data. I work on these topics full time and still find it impossible to keep up. Unless you are an even bigger tech nerd or privacy activist than me, the data landscape is simply too complex to navigate on your own. In most cases, data collection is an invisible process that happens behind the scenes and that—often by design—remains opaque to those whose data is being collected. Companies know what data they are harvesting, what they do with it, and how much it is worth to them. But you don’t.

For thousands of years, the most effective strategy to deal with situations of uncertainty was to look to others for guidance. If many other people followed a certain path or held a certain belief, it paid off to do the same. Privacy was no exception.

While growing up in Vögisheim, many of my early strategies for deciding what to disclose about myself and how to protect my personal life from the curious eyes of my neighbors were borrowed from what I observed in my friends and parents. Most of us do the same in today’s digital village. We don’t know how to deal with our personal data, so we look to others. Others, who in many cases are just as clueless as we are, and whose behaviors and beliefs are as easily swayed by those who want us to divulge as ours.

Let’s imagine our brains magically caught up. That would be fantastic, but still not enough. Managing our personal data would remain a full-time job. You simply don’t have the time to carefully read and decipher the legalese of all the terms and conditions you sign off on. Even the most efficient and diligent person only has twenty-four hours in a day. And (hopefully) better things to do than sifting through the terms and conditions of all the services and products they use. If you had to choose between sharing a meal with your family or deleting the browsing history on all your devices, which one would you pick?

The bottom line is that we cannot reasonably be expected to handle the responsibility that comes with the right to control our own personal data all by ourselves. Not in the current data ecosystem. There are simply too many forces working against us in this game.

Does all this mean we should give up on transparency and control altogether? Of course not. Tools like the original version of Tom’s app should exist for people to see what their data reveals about them.

And you should absolutely have control. It’s your data after all. But for us to be able to exercise control over the data successfully, we need to create systems that allow us to benefit from this right.

The Perfect Storm

Think of it this way: being the captain of a sailboat is easy and fun when you are drifting along the Mediterranean coast on a sunny day. You can choose any little town to visit. The one with the medieval cathedral or the one with the famous ice cream shop. There are no wrong choices. Now, imagine sailing the same boat through a raging thunderstorm. All by yourself. You could be thrown in any direction. There are twenty emergencies competing for your attention. Steering your boat under these circumstances doesn’t feel like a right at all.

Yet, that’s exactly what we do. We drop people in the middle of a raging technology storm—alone—and bless them with the right to control their personal data. That simply can’t be the optimal solution. We need more than transparency and control. We need to tame the sea with better regulation and staff our boat with a competent crew.

I’m going to start with the first part: How do we tame the sea and create an ecosystem that allows individuals to choose from a bright variety of desirable outcomes, rather than a dark mix of undesirable ones?