The two biggest revolutions in transport are electrification and autonomy. Those are the two biggest innovations since the moving production line and they are both happening at about the same time. The most near-term impact from a technology standpoint is autonomous cars, like fully self-driving cars. I think autonomy is extremely important. The car will be able to take you from point to point, like from your driveway to work without you touching anything. You could be asleep the whole time and do so very safely. That’s going to happen much faster than people realize. I think this is going to be quite a profound experience for people when they do it. We’ve been testing it for years so we got quite used to it, but I noticed when I put my friends in the car and they saw it drive itself they were blown away. It's really quite an interesting new experience. I think it's going to change people's perception of the future quite rapidly. It's a world-changing experience.
In the long term nobody will buy a car unless it is autonomous, it would be like having a manually operated elevator or something like that. There used to be elevator operators, you get in and there would be a guy moving a lever. Then we developed some simple circuitry to make elevators automatically come to the floor you are at, just press the button. Now you just get in and you press the button and that’s taken for granted, nobody needs to operate the elevator. It's a strange anachronism. The car is just going to be like that, getting in a car will be like getting in an elevator. You just tell it where you want to go and it takes you there with extreme levels of safety and that will be normal, that will just be normal. You'll be able to tell your car like, take me home, or go here, go there, anything, and it'll just do it, at an order of magnitude safer than a person.
I think it's going to be some time before one can truly just get in the car fall asleep and wake up at your destination, because you're going to have to take care of these corner cases. The expectations for an autonomous car safety will at least be a factor 10 higher then when a person is driving, because the moment a robot is driving and runs somebody over there would be big trouble.
The approach that we took is what we call Autopilot, because we think it's analogous to the way aircrafts operate, and after which our system is named. You still have pilots but you can engage in autopilot for most of the journey. With Autopilot version One the expectation was that someone’s attention is to the road and is ready to take over if there is an issue. It was really intended in the same way that autopilot for an aircraft works where autopilot for an aircraft alleviates workload but the pilot is still expected to pay attention, so he couldn't turn like Autopilot on and go to sleep.
If there's heavy snow it's going to be harder for the system to work. It's going to advise caution in heavy precipitation. It's a real boon in a high traffic situations. If you're in slow moving gridlock traffic, turn on Autopilot and it works super well, almost to the point where you can take your hands off. I won't say you can take your hands off, but almost. Our Autopilot capability is really good in heavy traffic, it’s super good in heavy traffic. Not that I'd recommend it but you can read a book or do email is what I've found ...err heard people say. It can really take the edge off the traffic.
Certainly in the long term people will not need hands on the wheel, and eventually there won't be wheels! There won't be wheels or pedals, it will just be... you jump in a car and go somewhere. You tell the car your destination and it will take you there. In order for that to occur it needs to be fail-operational. If any one system in the car fails for any reason, the car does not crash, that's still some ways away.
I think you'll probably want to have a steering wheel and pedals and be able to take control of the car when you want to take control. I don't super love the idea of having a bland little pod that you get in and go from one place to another in a very sort of conservative driving manner or something like that, it sounds boring. But it might be something like in ‘I, Robot’ where the car has an autonomous mode but you can switch to manual when you want to and the steering wheel comes out of the dash, that looked cool. Yeah, so I think autonomy default with optional manual is probably the good way to go. I think the quality of the ride is always gonna matter. Nobody wants to drive if you're sitting in stop-and-go traffic, that's boring. But if you're driving on a beautiful country road or along the seaside, then I think it feels wonderful to drive and you want to do that. I don't think cars are going to just become some boring utility.
We try to make the car behave as though it's a really good chauffeur, like a really good driver, not too conservative, not too aggressive. When the cars interact with each-other, I would think they would actually just be as though the car is interacting with a good human driver. Essentially it's like a person to some degree, how well can a person figure out what route they can take? In the beginning it's going to be not as good as a person in some ways and better in some ways. I think it's just going to be where it makes it easier and easier to drive and over time it will actually be better than a person. I mean long term it'll be way better than a person, imagine a system that has 8 cameras, radar, ultrasonics, and it's processing all that at the millisecond level, never gets tired, and it's never had anything to drink, and it's not arguing with someone in the car (hopefully!) so it's not distracted, and it's has this huge dataset. There's just no way, that would be like competing with eight human experts simultaneously. There's just no way one person is going to be better, you just don't have eyes in the back of your head. It will obviously be way better than a person long term. It will be safer than a person driving for all pedestrians as well as people in the car and other cars. In the distant future probably people may outlaw driving cars because it is too dangerous. You cannot have a person driving a 2 ton death-machine. All I'd say is that full autonomy is going to come a hell of a lot faster than anyone thinks it will. I think what we've got under development is going to blow people's minds. It blows my mind, so.
I think we're still on track for the coast-to-coast drive, being able to go cross-country from LA to New York by the end of 2017, fully autonomous. The thing that will be interesting is that I'm actually fairly confident it will be able to do that route even if you change the route dynamically. If you say it’s going to be really good at one specific route that's one thing, but it should be able to really be very good, certainly once you enter a highway, to go anywhere on the highway system in a given country. So it's not sort of limited to LA-New York, we could change it and make it Seattle-Florida, that day, in real time. Say you were going from LA to New York, and now go from LA to Toronto. Yeah, essentially November or December we should be able to go all the way from a parking lot in California to a parking lot in New York, no controls touched at any point during the entire journey. I believe we're still on track for that.*
Another way to think of it is as an added safety feature and the added safety in the long run becomes autonomous. It takes things to another level of safety, I do want to emphasize: this does not mean perfect safety. Perfect safety is really an impossible goal. It’s really about improving the probability of safety, that’s the only thing that is really ever possible. The thing to appreciate about vehicle safety is it’s probabilistic, I mean, there's some chance that any time a human driver gets in a car, that they will have an accident that is their fault, it’s never zero.
Really the key threshold for autonomy is how much better does autonomy need to be than a person before you can rely on it. There will never be zero fatalities, there will never be zero injuries. The world is a very big place and there’s a huge number of people and a huge number of circumstances. It’s really just about minimizing the probability of death not the illusion of perfect safety. It's just a question of refining the details of the technology, and bring that to market, and then improving the nines of probability. In order to have a self driving car you have to have many nines of reliability so it is 99.9999% is how good it needs to be. Let's say the first approximation you would want a self driving car to be is an order of magnitude safer than a human driven car. For self driving or Autopilot I think it should be held to a standard that's maybe 10 times higher than a person. If it is like 10 times safer there is no more doubt there's no more debate which one is safe. The real trick of it is not how do you make it work say 99.9 percent of the time, because if a car crashes one in a thousand times then you're probably still not going to be comfortable falling asleep, you shouldn't be, certainly. It's never going to be perfect no system is going to be perfect, but if you say the car is unlikely to crash in a hundred lifetimes, or a thousand lifetimes, then people are like, OK, wow, if I were to live a thousand lives, I would still most likely never experience a crash, then that's probably OK.
The system is getting better. I think it’s really quite unequivocal that Autopilot improves safety. Just looking at fatalities it’s at least 2.5 to 3 times more miles per fatality and that number is growing every day. I think that the autonomy system is likely to at least mitigate the crash, except in rare circumstances. The hardware and the software are not yet at the point where a driver can abdicate responsibility. That will come at some point in the future, but it is not the case today. These are still the early days. If there's an accident the driver of the car is liable, we’re very clearly saying that this is not a case of abdicating responsibility.
I would estimate that with the improved fleet learning and software I think we will end up probably 3 times safer than a car that isn’t on Autopilot that’s my guess, it’s not minor. What's sort of less visible to the outside are all the cases where the version One of Autopilot actually did a lot to mitigate the accident, so that the impact velocity went from being potentially fatal or severe injury to customer stepped out and walked away. The thing that we think is really quite powerful is the fact that it tends to reduce the impact velocity. There might still be an impact, but if it decreases the impact velocity from something that would have been fatal or caused critical injuries to something where you walk away from the car, that’s enormously helpful and that’s a really significant difference.
One of the ironies that we’ve seen is counter intuitive and a lot of people on the consumer watchdog sites and in some cases on regulatory sites have assumed that Autopilot accidents are more likely for new users, in fact it is the opposite Autopilot accidents are far more likely for expert users, it is not the neophytes, it’s the experts. They get very comfortable with it and repeatedly ignore the car’s warnings. It’s like a reflex. The car will beep at them, they tug the wheel, the car will beep at them, they tug the wheel, and it becomes an unconscious reflex action. So we will see half a dozen or more, sometimes as many as 10 warning in one hour continuously ignored by the driver. We really want to avoid that situation.
I think this is really going to make a difference, but I do want to emphasize that it’s not going from bad to good. I think it would be morally wrong to withhold functionalities that improve safety simply in order to avoid criticisms or for fear of being involved in lawsuits. I feel quite strongly that as soon as you have data that says that autonomy improves safety – even hypothetically 1 or 2 percent safer – there’s 1.2 million people dying from automotive accidents a year, one percent is 12,000 lives saved. I think things are already good, they are already better than if there wasn’t Autopilot. This is very important to appreciate. This is not going from bad to good. It’s going from good to I think great.
Cyber security is a huge concern, one of the biggest risks for autonomous vehicles is somebody achieving a fleet wide hack. We have to make super sure that a fleet wide hack is basically impossible and that if people are in the car, that they have override authority on whatever the car is doing. If the car is doing something whacky you can press a button that no amount of software can override that will ensure you gain control of the vehicle and cut the link to the servers. That's pretty fundamental. It is my top concern from a security standpoint to make sure that a fleet wide hack or any vehicle specific hack can’t occur. In principle if someone was able to hack the autonomous Teslas they could say -- just as a prank, say send them all to Rhode Island from all across the United States. Well, that would be the end of Tesla, and a lot of angry people in Rhode Island-- that's for sure. They have the same problem with cellphones. It's kind of crazy today that we live quite comfortably in a world that George Orwell would have thought is super crazy. We all carry a phone with a microphone that can be turned on any time really without our knowledge, with a GPS that knows our position, and a camera, and all of our personal information. We do this willingly and it's kind of wild to think that's the case. Apple and Google have the same challenge of making sure there cannot be a fleet wide hack or systemwide hack of phones, or a specific hack. That's a top concern, it’s going to become a bigger and bigger concern. Tesla is pretty good at software compared to other car companies. Within the car even if someone gained access to the car there are multiple subsystems that also have specialized encryption. The power-train for example even if someone would gain access to the power-train or the braking system. As the technology matures all Tesla vehicles will have the hardware necessary to be fully self-driving with fail-operational capability, meaning that any given system in the car could break and your car will still drive itself safely. It is important to emphasize that refinement and validation of the software will take much longer than putting in place the cameras, radar, sonar, and computing hardware. I do think it's going to be a bigger challenge to ensure security for the other car companies.
I should add a note here to explain why Tesla is deploying partial autonomy now rather than waiting until some point in the future. The most important reason is that when used correctly it is already significantly safer than a person driving by themselves, and it would therefore be morally reprehensible to delay release simply for fear of bad press or some mercantile calculation of legal liability. According to the 2015 NHTSA report, automotive fatalities increased by 8%, to one death every 89 million miles. Autopilot miles exceeded twice that number and the system gets better every day. It would no more make sense to disable Tesla's Autopilot, as some have called for, than it would to disable autopilot in aircraft. It is important to note that some number of people die every year by getting twisted up by their bed sheet, literally some number of people die every year by vending machines falling on them. There are those unusual situations that people die, but I don’t think anyone is saying that there should be no bed sheets or no vending machines. It’s just that you have these rare events occasionally – tragic – but if we were to eliminate all of them, we would be essentially limited to sitting at home on a pillow as the only thing you are allowed to do.
Even once the software is highly refined and far better than the average human driver, there will still be a significant time gap, varying widely by jurisdiction, before true self-driving is approved by regulators. I think we may see some jurisdictions giving the okay a lot sooner than others. I think the timeframe that we think it's ready and then the timeframe that regulators will approve differs, because we've got to present the data to them, they’ve got to think about it; then they've got to render a verdict. That can sometimes be a long process and it varies quite a bit by jurisdiction. When you think about like the global average fatalities, it's sort of somewhere around 60, one fatality every 60 million miles on a global basis. So if you're at 6 billion miles, you're 100 times the fatalities per mile. You really start to get quite statistically significant at that point, and it can make quite a strong argument, I believe, at that point that it would be morally wrong not to allow autonomous driving. I think the logical thing is that if there are fewer accidents in autonomous mode than in non-autonomous mode, there shouldn't be some penalty. That wouldn't make any sense you’d be penalizing a safer situation.
It's really hard for a person to compete. I mean, the car has eight cameras looking 360 degrees all the time, it’s got a forward radar, it’s got 12 high-precision ultrasonic sonars, it’s got initial measurement units, high-accuracy GPS, and over 10 Tera-ops of computing capability that never sleeps.
How does it figure out what to do? There's four major sensor systems. We've got the ultrasonic sensors, so essentially ultrasonic sonar, which tells us where everything is, so around the perimeter of the car we know where there are obstacles. That's then combined with the forward-facing camera with image recognition. The forward-facing camera is able to determine where the lanes are, where cars are ahead of it, and it's also able to read signs. Then that's combined with the forward radar. The radar is very good at detecting fast moving large objects. It can actually see through fog, rain, snow, and dust. The forward radar gives the car superhuman senses, it can see through things that a person could not. The exciting thing is that even if the vision system doesn’t recognize the object, because it could be a very strange-looking vehicle, it could be a multi-car pileup, it could be a truck crossing the road, it really could be anything – an alien spaceship, a pile of junk metal that fell off the back a truck, it actually doesn’t matter what the object is it just knows that there’s something dense that it is going to hit, and it should not hit that. It doesn’t need to know what that thing is – while a vision system really needs to know what the thing is. Radar sees through rain, fog, snow, dust, essentially quite easily. Even if you are driving down the road and the visibility was very low and there was a big multi-car pileup or something like that and you cant’ see it, the radar would initiate braking in time to avoid your car being added to the multi-car pileup.
Then the final sensor is the GPS with high precision digital maps. The high precision digital maps are important because normal maps have a quite low precision. Not only does it know where the street is, but the actual curvature of the road, how many lanes there are, and how you merge from one lane to the next. This is not present in any data set in the world but we're creating that data set at Tesla. This is all just in a statistical database, there's no user attribution. We don't know who it was, or when it was. We know that this is where a road exists. This is where cars have gone, statistically speaking.
Auto steer is using different visual cues, or visual road cues, to decide where to drive. It'll take the left lane, or right lane, depending upon where it is. So it's constantly looking up where it is in the world and depending upon its specific location it'll know whether to use the left lane marking, the right lane marking, follow vehicles, to use holistic path prediction or to go purely on navigation, on GPS.
We also have Side Collision Avoidance which is active all the time. You can turn it off, so this is not something you can't turn off, but it's separately turned off. It's like Automatic Emergency Braking, essentially. The Side Collision Avoidance what it will do is it will resist movement if you attempt to turn into another vehicle or into let's say a highway barrier without realizing it, you’ll feel increased resistance in the steering wheel. You feel like there's something unnatural here that's like... resisting. You can overcome it if you want to, but it's gonna tell you that you probably shouldn't move sideways, because you can sense that it's like harder on the steering wheel to move to one side or the other. It's a sort of general safety system like Automatic Emergency Braking.
Yeah, we feel highly confident that the 8-camera solution with 12 ultrasonics and a forward radar, and the computing power that we now have onboard is capable of full autonomy at a – it's simply greater than human. Like, you can probably do it ten times better than humans would, just cameras. You can absolutely be superhuman with just cameras. There are obviously skeptics out there. Well, I suggest that they do not bet against us.
While we are reaching the limit of the hardware I think we have not quite yet reached the algorithmic intelligence on the car, and of course anything that’s done on our servers we are not computer constraint or space constraint in any way.
Full autonomy is really a software limitation. Software is an increasing proportion of the problem particularly as you get to autonomy. I think we are a software and a hardware company, but the software component does become increasingly important. You got to get both right because it's a holistic product experience. I mean the hardware is just to create full autonomy, so it's really about developing advanced, narrow AI for the car to operate on. I want to emphasize narrow AI, it's like not going to take over the world, but it needs to be really good at driving a car. So increasingly sophisticated neural maps that can operate in reasonably sized computers in the car.
Something that quite uniquely Tesla is being able to do is fleet learning, I think this is quite a powerful network effect. Any car company that doesn't do this will not be able to have a good autonomous driving system. I think the big differentiator here is that the whole Tesla fleet operates as a network. When one car learns something, the whole fleet learns it. In order to have that, all the cars need to be connected. They need to be uploading data to a central server, where it can be collected, do statistical analysis on it, and then feed that back into the driving algorithm to the cars. That's I would say like a next-level technology, and certainly far beyond what any other car company is doing. I'm not sure they even are thinking about it. I've never heard them mention it, let me put it that way.
The thing that's quite interesting and unique is that we're employing a deep learning technology. Essentially, the network of vehicles is going to be constantly learning, and as we release the software and more people enable Autopilot, the information about how to drive is uploaded to the network. So each driver is effectively an expert trainer in how the Autopilot should work.
There is an enormous amount of sort of visual data being gathered. It's actually quite a challenge to process that data, and then train against that data, and have the vehicle learn effectively from data, because it's just a vast quantity of data. It’s both the data and the way that data is used by the car, what algorithms we use with the data. Those things are both improving rapidly over time. They have a multiplying effect, it’s sort of like the data multiply by the quality of the algorithms, and the data is increasing rapidly and the quality of algorithms increase rapidly. It’s really quite dramatic over time. We can use fleet learning to have all the Tesla cars out there effectively give us the geo-locations of where all the false alarm occurs, and what the shape of that object is that causes the false alarm. That we know that at a particular position at a particular street or highway, that if you see a radar object of a following shape – don’t worry it’s just a road sign or a bridge or it could be a Christmas decoration that somebody put across the street. You do need an additional overlay on that to understand turn restrictions, because you could certainly say, 'if the number of cars that turn left at an intersection is .5 percent of the time, then it's probably that they're just doing an illegal left.' so then we should ignore that. You'll see statistically if a turn is allowed or not allowed. I do want to emphasize that this is disaggregated from the specific vehicle. We're always on the side of the owner of the car and do whatever is possible within bounds of the law to protect privacy.
The data from the Tesla fleet can provide high precision information about routes. We certainly would be open to selling that to other car companies or other organizations if they want to buy it. It's really the fleet collectively that is producing this dataset. We're using that to provide high precision GPS navigation. It is kinda machine learning, with the drivers of the car essentially providing the training dataset, they’re training a collective fleet intelligence of all Teslas. It is an automatic learning system. To do autonomous driving to the degree that it's much safer than a person is much easier than people think.
I think really we want to ensure people think of their cars as connected devices. People should see the car actually improve probably with each passing week, even without a new software update, because the data is continually improving, and because the more miles that are driven the better the network intelligence of the fleet is trained the better it will get. It should actually get better with each passing day, but you'll probably notice it maybe after a week or a few weeks. You'll see that previously the car wouldn't have steered quite right, let's say going past a freeway offramp one week, but then the following week it does. That's really the way a car should operate, much in the way your laptop or your cell phone operates, that we can do improvements over-the-air.
It is important for safety and for improved functionality that carmakers in general go to a connected philosophy. So instead of; you issue a recall, and you don't always get all the cars coming in for various reasons, people lose touch with their dealer, and then they have unsafe software in their car but not know it because they didn't get the recall notice or weren't aware of it. That’s I think also just what consumers expect. It's rather odd to have a device that's not connected.
As far as gathering navigation data when there's no cell connectivity, the car can just buffer the data, and then upload the data once it gets to a place where there is cell connectivity or a WiFi connection. So there's no problem collecting data even when there's no cell phone connectivity. And of course the GPS satellites you can see all the time. Anywhere on Earth you can see the GPS satellites whether or not you have connectivity in that area, you can still drive on GPS functionality.
I'm really quite optimistic about where things are and where they're headed on that front. I think they're headed to a good place, but the perfect is the enemy of the good. You just can’t come fully formed into some ideal solution. It’s impossible to do that for anything. But as our fleet grows, and it's growing rapidly, it becomes clearer and clearer. The fleet learning will continue and the intelligence of how that fleet learning is applied to the car will continue to improve. It will continue to improve for years to come even with the existing hardware.
I think almost all cars built will be capable of full autonomy in about ten years. As it is the Tesla cars that are made today have the sensor system necessary for full autonomy and we think probably enough compute power to be safer than a person. It’s mostly just a question of developing the software and uploading the software. If it turns out that more compute power is needed, we can easily upgrade the computer.
I almost view it, and this may sound complacent, as a solved problem. We know exactly what to do and we will be there in a few years, and other manufacturers will follow and do the same thing. I think things are going to grow exponentially. Probably in ten years more than half of new vehicle production will be electric in the US, I think almost all cars produced will be autonomous in 10 years. It will be rare to find one that is not. I think it will be quite unusual to see cars in production that don't have full autonomy, let's say in the 15 to 20 year time frame, and for Tesla, it will be a lot sooner than that. I actually think at the point at which cars are being made that have full autonomy that any cars that are being made that don't have full autonomy will have negative value. I think the whole industry ultimately will be producing autonomous cars, but that’s not the same as all cars on the road.
It’s going to be a great convenience to be an autonomous car but there are many people who’s jobs it is to drive. In fact, I think it might be the single largest employer of people is driving in various forms. We need to figure out new roles for what do those people do, but it will be very disruptive and very quick. I should characterize what I mean by quick because ‘quick’ means different things to different people. It is important to just appreciate the size of the automotive industrial base. It's not as though when somebody makes autonomous cars suddenly all the cars will be autonomous. The global fleet of vehicles is about 2.5 billion roughly, and total new vehicle production capacity per year is only about 100 million, there's roughly 2.5 billion cars and trucks on the road and just under 100 million produced every year. So the production rate is only 5% of the fleet size. Which makes sense because the life of a car or truck before it’s finally scrapped is about 20, 25 years. The fleet is basically turning over every roughly 20, 25 years. I think the demand for autonomous cars will vastly outweigh the production capability. If tomorrow all cars were autonomous it would take 20 years to replace the fleet, assuming the fleet stays the same size. Arguably the fleet would get smaller if cars were autonomous. It's not all going to transition immediately it is going to take a while. It's the same for the electrification of cars.
The point at which we see full autonomy appear will not be the point at which there is massive societal upheaval because it will take a long time to make enough autonomous vehicles to disrupt employment. That disruption I’m talking about will take place over about 20 years, but still 20 years is a short period of time to have I think something like 12% to 15% of the workforce be unemployed.
The fundamental economic utility of a true self-driving car is likely to be several times that of a car which is not. Most cars are only in use by their owner for 5% to 10% of the day. There will be a shared autonomy fleet where you buy your car and you can choose to use that car exclusively, you could choose to have it be used only by friends and family, only by other drivers who are rated five star, you can choose to share it sometimes but not other times. That's 100 percent what will occur. Absolutely this is what will happen, it's just a question of when.
A lot of people think that when you make cars autonomous, they'll be able to go faster and that will alleviate congestion. To some degree that will be true, but once you have shared autonomy where it's much cheaper to go by car and you can go point to point, the affordability of going in a car will be better than that of a bus. Like, it will cost less than a bus ticket. The amount of driving that will occur will be much greater with shared autonomy and actually traffic will get far worse.
With the advent of autonomy it will probably make sense to shrink the size of buses and transition the role of bus driver to that of fleet manager. Traffic congestion would improve due to matching acceleration and braking to other vehicles, thus avoiding the inertial impedance to smooth traffic flow of traditional heavy buses. It would also take people all the way to their destination. Fixed summon buttons at existing bus stops would serve those who don't have a phone.
In cities where demand exceeds the supply of customer-owned cars, Tesla will operate its own fleet, ensuring you can always hail a ride from us no matter where you are. That's our focus. I'm very very optimistic about this. It's exciting, it blows me away, the progress we're making. I think if I'm this close to it and it's blowing me away, it's really going to blow other people away.
Autonomy will be widespread. It's going to be weird to have a car without it in the future. I think that, in the long term, owning a car that does not have autonomous capability will be a bit like owning a horse. You will only drive if you want to drive, but not for daily use really. You sort of own a horse for sentimental reasons, but not for actual transport.
*It is certainly possible that I may have egg on my face on that front. But if it is not, at the end of the year, it will be very close.