Ever since the region first emerged as an industrial zone after World War II, Silicon Valley has been reinventing itself. It was once known for microchips and mainframes. Then came personal computers and the web. These days, artificial intelligence looms large. Companies are investing heavily in AI and snapping up all the AI experts they can find. The next incarnation of Silicon Valley, it seems fair to say, will revolve around AI—if it doesn’t already.
But what even is AI? This should be a simple question, but honest answers are surprisingly hard to find. Mystification and misinformation abound, amplified by a media that’s typically far too deferential to industry hype. We’re told that AI is about to revolutionize everything—among other things, by throwing millions of people out of work by automating away their jobs.
This didn’t sound quite right to us, so we sat down with a veteran data scientist to learn more. The data scientist helped us sort the fact from the fiction, and obtain a clearer view of Silicon Valley’s next next big thing. When you strip away all the nonsense, what’s actually going on?
I would say the people who are the most confident about self-identifying as data scientists are almost unilaterally frauds. They are not people you would voluntarily spend a lot of time with.
There are a lot of people in this category who have only been exposed to a little bit of real stuff—they’re sort of peripheral. You actually see a lot of this with these strong AI companies: companies that claim to be able to build human intelligence using some inventive “Neural Pathway Connector Machine System,” or something.1 You can look at the profiles of every single one of these companies. There are always people who have strong technical credentials, and they are in a field that is just slightly adjacent to AI, like physics or electrical engineering.
And that’s close, but the issue is that no person with a Ph.D. in AI starts one of these companies, because if you get a Ph.D. in AI, you’ve spent years building a bunch of really shitty models, or you see robots fall over again and again and again. You become so acutely aware of the limitations of what you’re doing that the interest just gets beaten out of you. You would never go and say, “Oh, yeah, I know the secret to building human-level AI.”
In a way it’s sort of like my dad, who has a Ph.D. in biology and is a researcher back east, and I told him a little bit about the Theranos story.2 I told him their shtick: “Okay, you remove this small amount of blood, and run these tests…” He asked me what the credentials were of the person starting it, and I was like, “She dropped out of Stanford undergrad.” And he was like, “Yeah, I was wondering, since the science is just not there.” Only somebody who never actually killed hundreds of mice and looked at their blood—like my dad did—would ever be crazy enough to think that was a viable idea.
So I think a lot of the strong AI stuff is like that. A lot of data science is like that, too. Another way of looking at data science is that it’s a bunch of people who got Ph.D.s in the wrong thing, and realized they wanted to have a job. Another way of looking at it—I think the most positive way, which is maybe a bit contrarian—is that it’s really, really good marketing.
As someone who tries not to sell fraudulent solutions to people, it actually has made my life significantly better because you can say “big data machine learning,” and people will be like, “Oh, I’ve heard of that, I want that.” It makes it way easier to sell them something than having to explain this complex series of mathematical operations. The hype around it—and that there’s so much hype—has made the actual sales process so much easier. The fact that there is a thing with a label is really good for me professionally.
But that doesn’t mean there’s not a lot of ridiculous hype around the discipline.
As far as I know, the term “data science” was invented by Jeff Hammerbacher at Facebook.
Yeah, the Cloudera guy. As I understand it, “data science” originally came from the gathering of data on his team at Facebook.
If there was no hype and no money to make, essentially, what I would say data science is, is the fact that the data sets have gotten large enough where you can start to consider variable interactions in a way that’s becoming increasingly predictive. And there are a number of problems where the actual individual variables themselves don’t have a lot of meaning, or they are kind of ambiguous, or they are only very weak signals. There’s information in the correlation structure of the variables that can be revealed, but only through really huge amounts of data.
So essentially, there are n variables, right? So there’s n-squared potential correlations, and n-cubed potential cubic interactions or whatever. Right? There’s a ton of interactions. The only way you can solve that is by having massive amounts of data.
You can start to see new things emerge that would not emerge from more standard ways of looking at problems. That’s probably the most charitable way of putting it without any hype. But I should also say that the hype is just ferocious.
And even up until recently, there’s just massive bugs in the machine-learning libraries that come bundled with Spark.4 It’s so bizarre, because you go to Caltrain [Bay Area commuter rail line], and there’s a giant banner showing a cool-looking data scientist peering at computers in some cool ways, advertising Spark, which is a platform that in my day job I know is just barely usable at best, or at worst, actively misleading.
I don’t know. I’m not sure that you can tell a clean story that’s completely apart from the hype.
It’s a really good question. I’m not even sure if those terms that you referenced are on solid ground themselves.
I’m friends with a venture capitalist who became famous for coining the phrase “machine intelligence,” which is pretty much just the first word of “machine learning” with the second word of “artificial intelligence,” and as far as I can tell is essentially impossible to distinguish from either of those applications.
I would say, again, “data science” is really shifty. If you wanted a pure definition, I would say data science is much closer to statistics. “Machine learning” is much more predictive optimization, and “AI” is increasingly hijacked by a bunch of yahoos and Elon Musk types who think robots are going to kill us. I think “AI” has gotten too hot as a term. It has a constant history since the dawn of computing of overpromising and substantially underdelivering.
They think of the film Artificial Intelligence, that level of AI, yeah. And as a result, I think people who are familiar with bad robots falling over shy away from using that term, just because they’re like, “We are nowhere near that.” Whereas a lot of people who are less familiar with shitty robots falling over will say, “Oh, yeah, that’s exactly what we’re doing.”
Can I say the truth is halfway there? I mean, again, I want to emphasize that historically, from the very first moment somebody thought of computers, there has been a notion of, “Oh, can the computer talk to me, can it learn to love?” And somebody, some yahoo, will be like, “Oh, absolutely!” And then a bunch of people will put money into it, and then they’ll be disappointed.
And that’s happened so many times. In the late 1980s, there was a huge Department of Defense research effort toward building a Siri-like interface for fighter pilots. And of course this was thirty years ago and they just massively failed. They failed so hard that DARPA was like, “We’re not going to fund any more AI projects.”6 That’s how bad they fucked up. I think they actually killed Lisp as a programming language—it died because of that. There are very few projects that have failed so completely that they actually killed the programming language associated with them.
The other one that did that was the—what was it, the Club of Rome or something?7 Where they had those growth projections in the 1970s about how we were all going to die by now. And it killed the modeling language they used for the simulation. Nobody can use that anymore because the earth has been salted with how shitty their predictions were.
Yes, exactly, or the name Adolf. Like, you just don’t go there.
So, I mean, that needs to be kept in mind. Anytime anybody promises you an outlandish vision about what AI is, you just absolutely have to take it with a grain of salt, because this time is not different.
I think it’s irrelevant in our lifetimes and in our grandchildren’s lifetimes. It’s a very good philosophical question, but I don’t think it really matters. I think that we are going to be stuck with specific AI for a very, very long time.
Optimization around a specific problem, as opposed to optimization on every problem.
Yeah. Whereas if we invented a brain that we can teach to do anything we want, and we have chosen to have it focus on the specific vertical of driving a car, but it can be applied to anything, that would be general AI. But I think that would be literally making a mind, and that’s almost irresponsible to speculate about. It’s just not going to happen in any of our lifetimes, or probably within the next hundred years. So I think I would describe it as philosophy. I don’t know, I don’t have an educated opinion about that.
I would say that robo-advisers are not doing anything special. It’s AI only in the loosest sense of the word. They’re not really doing anything advanced—they’re applying a formula. And it’s a reasonable formula, it’s not a magic formula. They’re not quantitatively assessing markets and trying to make predictions. They’re applying a formula about whatever stock and bond allocations to make—it’s not a bad service, but it’s super hyped. That’s indicative of a bubble in AI that you have something like that where you’re like, “It’s AI!” and people are like, “Okay, cool!”
There’s a function that’s being optimized—which is, at some level, what a neural net is doing.9 But it’s not really AI.
I think one of the big tensions in data science that is going to unfold in the next ten years involves companies like SoFi, or Earnest, or pretty much any company whose shtick is, “We’re using big data technology and machine learning to do better credit score assessments.”10
I actually think this is going to be a huge point of contention moving forward. I talked to a guy who used to work for one of these companies. Not one of the ones I mentioned, a different one. And one of their shticks was, “Oh, we’re going to use social media data to figure out if you’re a great credit risk or not.” And people are like, “Oh, are they going to look at my Facebook posts to see whether I’ve been drinking out late on a Saturday night? Is that going to affect my credit score?”
And I can tell you exactly what happened, and why they actually killed that. It’s because with your social media profile, they know your name, they know the names of your friends, and they can tell if you’re black or not. They can tell how wealthy you are, they can tell if you’re a credit risk. That’s the shtick.
And my consistent point of view is that any of these companies should be presumed to be incredibly racist unless presenting you with mountains of evidence otherwise. Anybody that says, “We’re an AI company that’s making smarter loans”: racist. Absolutely, 100 percent.
I was actually floored, during a recent Super Bowl, when I saw this SoFi ad that said, “We discriminate.” I was just sitting there watching this game, like, I cannot believe it—it’s either they don’t know, which is terrifying, or they know and they don’t give a shit, which is also terrifying.
I don’t know how that court case is going to work out, but I can tell you in the next ten years, there’s going to be a court case about it. And I would not be surprised if SoFi lost for discrimination. And in general, I think it’s going to be an increasingly important question about the way that we handle protected classes generally, and maybe race specifically, in data science models of this type.11 Because otherwise it’s like, okay, you can’t directly model if a person is black. Can you use their zip code? Can you use the racial demographics for the zip code? Can you use things that correlate with the racial demographics of their zip code? And at what level do you draw the line?
And we know what we’re doing for mortgage lending—and the answer there is, frankly, a little bit offensive—which is that we don’t give a shit where your house is. We just lend. That’s what Rocket Mortgages does.12 It’s a fucking app, and you’re like, “How can I get a million-dollar loan with an app?” And the answer is that they legally can’t tell where your house is. And the algorithm that you use to do mortgages has to be vetted by a federal agency.
That’s an extreme, but that might be the extreme we go down, where every single time anybody gets assessed for anything, the actual algorithm and the inputs are assessed by a federal regulator. So maybe that’s going to be what happens. I actually view it a lot like the debates around divestment. You can say, “Okay, we don’t want to invest in any oil companies,” but then do you want to invest in things that are positively correlated with oil companies, like oil field services companies? What about things that in general have some degree of correlation? How much is enough?
I think it’s the same thing where it’s like, okay, you can’t look at race, but can you look at correlates of race? Can you look at correlates of correlates of race? How far do you go down before you say, “Okay, that’s okay to look at”?
That’s right. And the part that I find offensive as a mathematician is the idea that somehow the machines are doing something wrong. We as a society have not chosen to optimize for the thing that we’re telling the machine to optimize for. That’s what it means for the machine to be doing illegal things. The machine isn’t doing anything wrong, and the algorithms are not doing anything wrong. It’s just that they’re literally amoral, and if we told them the things that are okay to optimize against, they would optimize against those instead. It’s a frightening, almost Black Mirror–esque view of reality that comes from the machines, because a lot of them are completely stripped of—not to sound too Trumpian—liberal pieties. It’s completely stripped.
They are massively not politically correct, and it’s disturbing. You can load in tons and tons of demographic data, and it’s disturbing when you see percent black in a zip code and percent Hispanic in a zip code be more important than borrower debt-to-income ratio when you run a credit model. When you see something like that, you’re like, Ooh, that’s not good. Because the frightening thing is that even if you remove those specific variables, if the signal is there, you’re going to find correlates with it all the time, and you either need to have a regulator that says, “You can use these variables, you can’t use these variables,” or, I don’t know, we need to change the law.
As a data scientist I would prefer if that did not come out in the data. I think it’s a question of how we deal with it. But I feel sensitive toward the machines, because we’re telling them to optimize, and that’s what they’re coming up with.
Yeah. That’s right, that’s right. That’s exactly what they’re doing. I think it’s scary. I can tell you that a lot of the opportunity those fintech companies are finding is derived from that kind of discrimination, because if you are a large enough lender, you are going to be very highly vetted, and if you’re a very small lender you’re not.13
Take SoFi, for example. They refinance the loans of people who went to good colleges. They probably did not set up their business to be super racist, but I guarantee you they are super racist in the way they’re making loans, in the way they’re making lending decisions.
Is that okay? Should a company like that exist?
I don’t know. I can see it both ways. You could say, “They’re a company, they’re providing a service for people, people want it, that’s good.” But at the same time, we have such a shitty legacy of racist lending in this country. It’s very hard not to view this as yet another racist lending policy, but now it’s got an app.
Fintech may most accurately be described as regulatory arbitrage: startups are picking up pieces that a big bank can’t do, won’t do, or that are just too small for it to pick up. And I think fintech is going to suffer over the next five years. If there’s a single sector that people are going to be less enamored with in five years than they are now, fintech is definitely the one.
The other side of it is that they’re exploiting a hack in the way venture capitalists think. Venture capital as an industry is actually incredibly small relative to the financial system. So if you were starting, I don’t know, a company that used big data to make intelligent decisions on home loans—which is probably illegal, but whatever, you’re small enough that it’s no big deal—and you say, “Hey, we’re doing ten million dollars a year in business,” a venture capitalist will look at you like, “Holy shit, I’ve never seen a company get up to ten million dollars in business that fast.” The venture capitalist has no idea that the mortgage market is worth trillions of dollars and the startup essentially has none of it. The founder gives a market projection like, “Oh, this is a trillion-dollar industry,” and the venture capitalist is like, “Oh, that market is enormous. I’ve never seen numbers like that before.”
It’s much more of a clever hack than an actual, sustainable, lasting, value-creating enterprise. One of the biggest flagship fintech companies, LendingClub, is in a ton of trouble.14 SoFi is probably illegal. And those are the flag bearers for the sector.
The other thing that happened was the San Bernardino shootings—apparently the guns that were used were financed by a loan from Prosper, which is another peer-to-peer lender.15 And you just think about where this is going to go. Are we eventually going to get to the point where we have the credit models to assess and not give that guy a loan because of the risk that he could be a Muslim terrorist? Is that the society that we will be living in?
Maybe. But we’re going to get there with the data.
I think the most realistic way of looking at it is that it’s not all hype. The technical advances are real—and even if they’re not real today, the relentless drumbeat of progress on hardware and algorithms will make them real eventually. It will take longer than you think—potentially a lot longer than you think—but it will happen. So everything you’re hearing is an early warning sign of what the future is going to look like. Maybe not even in our lifetimes, but yeah, it’ll get there. And the questions that we’re going over, they’re going to be real. It’s just not there yet.
So yeah, I would recommend some skepticism, but not complete skepticism. Because the advances underlying this are real. And the rate of progress has kept up, and I don’t see a reason why it’s going to stop.
It’s funny, because I think the biggest concern that I have for the future is that a bunch of people like me are going to make a bunch of money. And a bunch of people are going to lose their jobs. And a bunch of people are going to get new jobs that are crazy and cool. But I don’t know on net how great it’s going to be for society moving forward, though I want to be optimistic about it.
I mean, I try to not fall into the Y Combinator/Stanford guy thing, but I actually do think that universal basic income is going to be the endgame.16 I think that is what society will look like long-term, because I think universal basic income is the welfare that everyone can get behind.
But it’s such a weighty question, and technology’s impact on the economy changes so quickly that I don’t know if any of us have ever really had the chance to take a breath. You look at some of the strikes a hundred years ago, like at the Homestead plant, where the workers held out and had fucking gunboats come down the river with Pinkertons and shoot the shit out of people.17 There’s a Costco there now, and a bunch of smokestacks where the plant used to be. And it’s like, was that whole thing just this ridiculous farce? I don’t even know.
I think the strangest thing about being out here in the Bay Area is that the worldview has just completely saturated everything to the point that people think that everything is a technical problem that should be solved technologically. It’s a very privileged view of very smart people. It’s troubling.
On the one hand, there’s no better shepherd for the economy than an engineer; on the other hand, there’s no worse shepherd for the economy than an engineer. Because that kind of machine thinking is very good at producing some things, and very, very bad at producing other things.
On the one hand, I don’t view any of the Silicon Valley startup economies as producing any kind of sustainable growth or ways of employing all these people. On the other hand, I do think that the basic income idea eventually will be the future. One of the most interesting things is the amount of leverage that individual people in Silicon Valley are getting—you look at the WhatsApp acquisition or whatever, with so few people being worth so much money.
That may have been a little bit irrational, but longer term, it’s hard to argue against. And I don’t see another endgame other than pretty high taxes plus basic income as the way of making that okay, because I don’t think that’s going to go away. I’m not even totally sure that we should discourage it from happening.
I think both approaches are connected to this overwhelming need to see political problems as technical ones, whether from an engineering perspective or from a technocratic governance perspective. To me those feel totally compatible. What you’re describing—the Silicon Valley view of the world—feels to me like a very technocratic view of the world, where if you can just solve certain problems, then it will benefit everyone.
In defense of it, it’s also a hopeful view of the world, because you’re at least trying to describe problems that you can solve. It’s a very optimistic way of looking at things, and I’m hesitant to abandon that, because I think ultimately … it’s hard, grappling with this idea of the enormous amount of individual leverage and the crazy rate of change.
On the other hand, it’s hard not to be a kind of technical utopian. It’s hard to bet against the innovation that this country has produced, and maybe that’s a function of survivorship bias or looking back and saying we just happened to get lucky. But you know, airplanes, the elevator—we just invented that stuff, and that’s kind of cool. And so it seems sort of melancholy—or maybe this is my own limitation as a technical thinker to see it as melancholy—to be like, “Yeah, there’s some stuff we can’t solve.”
I’m not sure I want to live in that world. I always want to live in a world where we’re at least trying. But we’ll see.