A Smarter Vehicle Is a Safer Vehicle
December 3, 2019
Sponsored by Arm
Self-driving technology aims to eliminate human error—the leading cause of automobile accidents. We traveled to the heart of Silicon Valley to discuss the smart solutions enhancing self-driving technology. In this episode sponsored by Arm, experts explain the cloud, AI and IoT innovations keeping riders safe.
Guests
-
Phil Magney
A Smarter Vehicle Is a Safer Vehicle - CES Tech Talk Podcast
Self-driving technology aims to eliminate human error—the leading cause of automobile accidents. We traveled to the heart of Silicon Valley to discuss the smart solutions enhancing self-driving technology. In this episode sponsored by Arm, experts explain the cloud, AI and IoT innovations keeping riders safe. -
Mark Douglas
A Smarter Vehicle Is a Safer Vehicle - CES Tech Talk Podcast
Self-driving technology aims to eliminate human error—the leading cause of automobile accidents. We traveled to the heart of Silicon Valley to discuss the smart solutions enhancing self-driving technology. In this episode sponsored by Arm, experts explain the cloud, AI and IoT innovations keeping riders safe. -
Robert Day
A Smarter Vehicle Is a Safer Vehicle - CES Tech Talk Podcast
Self-driving technology aims to eliminate human error—the leading cause of automobile accidents. We traveled to the heart of Silicon Valley to discuss the smart solutions enhancing self-driving technology. In this episode sponsored by Arm, experts explain the cloud, AI and IoT innovations keeping riders safe.
Accordion
-
Tyler Suiters
This special edition of CES Tech Talk is brought to you by Arm. Arm Technology is at the heart of a computing and connectivity revolution; one that's transforming the way people live and businesses operate. Arm's advanced, energy efficient processor designs have been enabled intelligent computing in more than 150 billion chips, and its technologies now securely power products from the sensor to the smartphone and the car.
Tyler Suiters
Hey, everybody. With the Consumer Technology Association, I'm Tyler Suiters. We are the owners and producers of CES, the world's largest, most influential tech event. And we're here to help you get CES Ready. The show is January 7th through the 10th, 2020, in Las Vegas. And today, we are talking about self-driving vehicles, a critical innovation for our health and welfare. Consider this: About 95% of car crashes in the US today are caused by human error. So, what happens if self-driving vehicles can fully eliminate human error from that equation? 37,000 people are killed in the US in these kinds of accidents every year, and you move that out globally; we're talking about the potential to save more than 100 million lives worldwide.
Tyler Suiters
At CES, you will get ride and drive experiences and ride and no-drive experiences. You can test technologies that support the future of self-driving vehicles. And that includes technologies that are evolving even as we speak: parking assist, collision avoidance, emergency braking, and much more. And so today, a focus on self-driving vehicles; not so much the cars themselves, but all that goes into making them smart the artificial intelligence, the software involved.
Tyler Suiters
So a conversation with three different players in this field, all at once, too. First of all, Arm. This is a company that's providing the IP for chips. That means delivering cloud services and providing a complete IoT solution. Remarkable innovation, here.
Tyler Suiters
Also, NXP. This provides a clear, streamlined approach. It's a domain-based architecture, so we're talking about software and stack, et cetera. This is something that can intelligently group together the functions that allow cars to sense, to think, and to act very quickly. We're talking about managing complexity and separate concerns in almost real-time. And often, in real-time.
Tyler Suiters
And finally, VSI Labs, a top advisor that supports R&D and planning departments both in major auto companies and suppliers, as well. It is technical and applied research across hardware, software connectivity systems, all to advance SDV tech.
Tyler Suiters
So today, a deep drive brought to you by Arm, into the world of self-driving vehicle innovation.
Tyler Suiters
For this special edition of CES Tech Talk, we are in the heart of the self-driving vehicle world, Silicon Valley, California, talking to three experts on this topic. Phil Magney is Founder and Principal Advisor at VSI Labs. Mark Douglas is Field Software Solutions Architect at NXP, and Robert Day is Director of Automotive Solutions and Platforms at Arm. And gentleman, pleasure to have you all with us today. Thank you.
Phil Magney
It's a pleasure being here.
Tyler Suiters
So this is a road trip for us, this is your hometown. Robert, let's start with you. We're looking ahead to the future of self-driving vehicles. What, how, and when? I mean, what will they look like on the road beyond what we're seeing today, and when do we reach, oh, I don't know, I'll say critical mass, but maybe there's a better term for it?
Robert Day
Okay so the first ones are going to look like regular cars, except they're going to be strewn with sensors and bits hanging off them. But the guts of the car are going to be very different. So the mechanics that we, as users, use to actually steer and press the brakes and accelerate, they're going to be the same. That's going to be based on platform that's well-proven, is safe. The rest of it's all going to be compute and software. And that's where it gets really interesting.
Tyler Suiters
So the parts we can't see?
Robert Day
The bits you can't see.
Tyler Suiters
Mm-hmm (affirmative).
Robert Day
So the sensors, you can see. So you'll be able to see there's cameras. You'll be able to see there's lidars. You'll be able to see there's radars, so you'll know it's self-driving.
Tyler Suiters
Uh-huh (affirmative).
Robert Day
But the bits that are actually doing all the real work, the heavy lifting, are going to be in the car.
Mark Douglas
Yeah, and what's happening underneath all that is a competition for coulombs; for electricity.
Robert Day
Yeah.
Mark Douglas
Because we're going to have a completely different thing on our hands. And we also have the evolution of electric vehicles, as well. So, we'll see a combination of these things being powered electrically, but also driven through a central compute or a distributed compute, and we're seeing the evolution of those architectures at the same time.
Mark Douglas
So, there's a different kind of thing going on inside the vehicle that people aren't aware of, and they really don't need to be aware of. It's just going to work. Because a lot of people don't care how it gets done, just that it gets done.
Robert Day
But what is going to freak people out is there's not going to be a driver.
Mark Douglas
Yeah.
Tyler Suiters
Mm-hmm (affirmative).
Robert Day
That's the bit that's going to really wind people up, because they can't see the compute, they can just see there's no driver.
Phil Magney
I'd like to point out one other thing that's not very noticeable to looking at these cars, is the fact that they all have one thing in common, and that's a BiWire architecture. So they are... You have the ability to digitally actuate steering, throttle and brake. This is something that is completely invisible, but it is the fundamental foundation from which an automated vehicle technologies are built on.
Tyler Suiters
Yeah.
Mark Douglas
And a lot of that's been there for the longest time. Even in your regular vehicles, you're shifting, you're doing something and you're not really moving anything mechanical. It's already actuating through different channels. You're not actually doing a shift. You know? It's very interesting, there.
Robert Day
So what's interesting is the guts that they can't see is basically a ton of different computes with loads of software. And by loads, we're talking hundreds of millions of lines of software.
Tyler Suiters
Yeah, just exponentially more than you're dealing with right now, in a 2020 car, say.
Robert Day
Absolutely. Absolutely.
Tyler Suiters
So how difficult, or how profound are those challenges? Not just the iteration, the evolution of, let's call it the hardware, but the body of the car. Right? Making it more aerodynamic, more energy efficient, safer, all the things that go into constant vehicle development; but is what you don't see. That's where, I'd imagine, from your world, that's where the challenges really lie.
Phil Magney
Yeah, I think as Robert pointed out, I mean, you can certainly spot a development vehicle on the roads here in Silicon Valley because they do have massive arrays of sensors on the roof and a highly exposed, a lot of bits and pieces hanging off, as Robert said. And if you get the pleasure of looking in the trunk, you would even be really taken aback, because the things are loaded with computers and you have a massive power consumption going on with those things, as well. I mean, that's the difference, I would say, where the gaps lie between today's development vehicles and future vehicles that are commercialized, is that that stack, that hardware and software stack has to come down, be hardened and put into a platform that is really feasible and practical [crosstalk 00:07:29].
Mark Douglas
Yeah, so if we could itemize those, the safety element both in the hardware and software, the power budget for the whole vehicle and where all that's coming from [crosstalk 00:07:39].
Tyler Suiters
Cue the eye roll, if you can.
Mark Douglas
I know. Rolling—
Tyler Suiters
[crosstalk 00:07:41] joining us from the studio.
Robert Day
And head roll.
Mark Douglas
And head roll. Yeah, I know.
Tyler Suiters
Head roll from Mark, too.
Mark Douglas
I embellished that one. But those are the things that, when I go in to talk to customers, they're mainly either not aware of certain parts of that, but they are acutely aware of the power demands and what's going to happen and how much compute they have. Because what they started with in the POC, the proof of concept, was anything that they had available. A big PC, they'd ram it in the back of the vehicle and then some AI accelerator component that's also super coulomb-hungry and it's munching those things down. And where are they coming from? Well, they're driving in a traditional vehicle, and all that, when we go to manufacturing, which was a great premise for a question is, all that has to be changed. All that, that whole paradigm has to change to lower power consumption, all of those things we talked about; being safe, being verifiable in the software and the hardware.
Phil Magney
And likewise, I want to add, too, Mark, to build on that.
Mark Douglas
Oh, yeah. Sorry.
Phil Magney
And the sensors. I mean, they're all highly exposed right now. I mean, they are very, very susceptible to inclement weather. In fact, these vehicles will not operate in inclement weather without solutions designed to accommodate, to keep those sensors clean and dry. And so you'll start seeing a tighter integration, more various places within the vehicle hidden rather than being totally exposed like they are now.
Mark Douglas
Yeah. And Phil, coming from Minnesota, I'd assume that inclement weather has a different definition for you than it does for us out here.
Phil Magney
Absolutely, yeah. We are probably the only company in the state of Minnesota that is developing automated vehicles, but we're not out to build a better self-driving car. We run a testing lab, and our job is to really, with our own development vehicles, test out different combinations of sensors and computers and software. And obviously, a lot of the experiments we do are in the winter, so we are showing and demonstrating how you can operate these and cope with the inclement weather.
Tyler Suiters
Robert, I want you to play, if you would, off of something that Mark said regarding the design, and it all changing.
Robert Day
Mm-hmm (affirmative).
Tyler Suiters
And you were heading in this direction earlier, which is the way the car looks. Not just on the outside, but really on the inside. Because when you don't need to drive, why do you need four seats facing forward? Why do you need a cabin that has a specific design to it?
Robert Day
Yeah. And so it's very interesting. If you start positioning yourself from being a driver to a passenger, the inside of the car just changes completely. I actually don't want to know what speed we're doing. Okay? I really... I kind of almost don't—
Mark Douglas
It might become a mobile office, right?
Robert Day
Well, maybe, yeah. So I almost don't want to see what's going on around me. I'm just going to assume to... So it's like being in an airplane. You're assuming that the pilot's going to get you there, and you really don't want to know what's kind of going on. And so I think in a self-driving car, there'll be all of these things in there which will enable us to be in our home or in our office, enable us to keep working or keep playing, or whatever we want to do. And it's almost a distraction from what's going on in the car.
Robert Day
So I don't think there's going to be beeps and buzzes. It's not like KITT with Knight Rider, where it says, "Okay, Michael, I'm going to take over now," and off we go. It's really going to be, "I'm just going to get you from A to B. And all you're going to do is," like with your Uber app, "Is you type in, I want to get downtown," and it will take you downtown, and it will say, "You've arrived. Get out."
Tyler Suiters
Yeah. And far safer than we would ever do it ourselves.
Robert Day
Hell yeah.
Tyler Suiters
Mark, Phil brought up an interesting term, one I hadn't heard before: hardening the stack.
Mark Douglas
Yeah.
Tyler Suiters
You live, I believe, in the software stack.
Mark Douglas
That's true.
Tyler Suiters
Right?
Mark Douglas
Yeah. Oh, and the underlines, too.
Tyler Suiters
So—
Mark Douglas
The boot loaders, the kernel running underneath, and all of that stuff.
Tyler Suiters
Well, give us a view of what your life is like right now.
Mark Douglas
Sure.
Tyler Suiters
Not personally, but more about the professional space, because that's where you all are in consensus, here.
Mark Douglas
Okay.
Tyler Suiters
That's where the magic's happening.
Mark Douglas
Okay, let me bring out a few elephants in the room right now, is... Not in this room, but in the actual virtual room, if you will, of what's going on, is you have sort of an evolution in the software realm, where people have taken traditional operating systems like Linux and other POSIX-type runtime code. And on top of that, they've layered stuff that needs to keep track of all these sensors, timestamp it, do it in a very quick fashion, and isolate that environment. And then you have maybe multiple application spaces that are going to kind of come up in the future. We're seeing the evolution of a software-defined vehicle right now. That's one elephant in the room. And then also, this whole latency associated with time stamping items and then comparing them and making sure you're not comparing stale data from one to the other.
Mark Douglas
I would say the other elephant in the room is, there's a different philosophy of use. There's a different POU that comes from different manufacturers. And you guys all interact with... I'm looking at the rest of the folks, here. We all interact with people that have different ideas about it. Folks that are in areas with weather, they know they need lidar, radar, all the different types of sensors combined with camera. There's folks from another school of thought that believe that they can do the whole thing with just cameras and stereo cameras and changing different things. And we're starting to see that that, combined with the evolution of people trying to make the neural nets learn and be more accurate with less information or train them in a different fashion. So there's a whole bunch of different things that feed into this system to make it more efficient and better, and so on.
Mark Douglas
So I just threw out a whole bunch of stuff that the guys can work with, if they want to.
Phil Magney
Yeah, let me just add, based on the use cases, it's something that I like to always point out. When you're talking about automated vehicle technologies or driverless technologies, whatever label you want to call that, I mean, you really have to look at the different use cases. And there are kind of some primary ones. For example, you have series production automated technologies now coming into the market, level 2, level 2+, eventually level 3. And so that's one trajectory. And OEMs and developers are often working on that path by itself. You know? Because that's so unique. You know?
Phil Magney
And then you have the other companies working on the robo-taxis, which are, of course, your level 4 vehicles.
Tyler Suiters
Right.
Phil Magney
And those are the ones we were talking about earlier, fantasizing about having everything in the interior, being able to do what you want to do and so forth.
Tyler Suiters
Not fantasizing—envisioning.
Phil Magney
Exactly. And then, of course, you have the low-speed shuttles which are now a very pragmatic use case for automated vehicles that operate on campuses or communities or low-speed operating kind of people movers. So you got to kind of take a look at the different trajectories of it to really understand what's going on.
Robert Day
So I'd like to kind of come back to your initial question of when, how, what, where.
Tyler Suiters
That's an open-ended question, Robert, so you can hit that anytime.
Robert Day
Oh, okay. All right. Well, we'll keep going on that. So one of the interesting things is coming off something Phil just said, is what these initial deployments are going to look like, is they're going to be very specific use cases, or operational design domains, ODDs, they're called. And it's going to be, "Well, I'm going to be in this part of the city, I'm going to go at this speed. I'm only going to go out when it's daytime, only in this weather." And that defines how the car is going to operate.
Tyler Suiters
Mm-hmm (affirmative).
Robert Day
Then what you can do, is you can kind of laser in on, "Well, if I know I'm only going to be here, I'm going to map the heck out of it. I'm going to have these sort of sensors, I'm going to know where I am at any one time." Okay? And I can update that information just because I've got a very specific place I'm going to go.
Tyler Suiters
Mm-hmm (affirmative).
Robert Day
And so that will enable these first self-driving cars to actually hit the roads and hit the roads without having the safety driver. And the safety driver is the bit right now which is basically making sure that these things operate correctly because they can take over at any time.
Tyler Suiters
Mm-hmm (affirmative).
Phil Magney
Highly constrained operating design domain, I completely agree with that. I think it'll even be... It might even go further than that. It might be specific roads, specific lanes, specific times of day.
Robert Day
Yeah.
Phil Magney
You might even have predetermined drop off and pickup points.
Robert Day
Yep.
Phil Magney
I mean, the whole idea here is to limit the exposure of that vehicle to, so-called, edge cases or things that are outside of your control.
Tyler Suiters
Well, humans.
Phil Magney
Yeah.
Tyler Suiters
I was going to say, humans make important decisions, right? Yeah.
Mark Douglas
But there'll be plenty of redundancy in there. I mean, there'll be plenty of safety checkers in there. There'll be traditional elements always looking and grabbing and tapping off of the instructions that go to the drive by wire system. Because if you're going around a turn at 30 miles an hour, and the whole system says you should accelerate, depending on the weather and stuff and what you're measuring with the inertial systems and such, you're not going to want to do that, and something needs to catch that. And that'll always be there. In fact, a lot of the designs right now, they don't worry so much about the safety of the whole system, because they have a checker that's at [inaudible 00:16:37] level that's actually checking.
Mark Douglas
So a lot of people that, when we come in and talk to them, they're not yet transitioning in their brains. They're starting to a little bit, that they need to certain amount of that in their main system, but they're going, "Ah, we don't worry about that, yet, because we've got something always watchdogging and taking a look at that."
Phil Magney
So Mark, is this a deterministic wrapper, then, basically; the checker that essentially says, "Okay, there are certain things we physically can and cannot do under certain conditions," basically?
Mark Douglas
Mm-hmm (affirmative). That's right.
Phil Magney
And I think there've been several companies that have talked about their own checker solutions, if you will.
Mark Douglas
That's right.
Phil Magney
I won't get into names, but—
Tyler Suiters
It almost sounds like... And this is probably a ham-handed way to put it, but like geofencing.
Robert Day
Yes. [crosstalk 00:17:17] It's exactly geofencing.
Tyler Suiters
Boy, thanks for the validation, Robert.
Robert Day
You're welcome!
Tyler Suiters
Wow. [crosstalk 00:17:22] So Mark, you touched on the idea of responsibility; the redundancy of having a backup driver.
Mark Douglas
Yes, yes.
Tyler Suiters
Another point that comes to mind that you took a step in this direction, but responsiveness. And for the last, what, 50 years, we've talked about cars and responsiveness. You're talking about a sports car.
Mark Douglas
That's right.
Tyler Suiters
And how quickly can corner, how well it moves.
Mark Douglas
Oh, and those are the things that we like to drive, we like to do. This whole thing is going to be very jarring because we're taking that element away from folks, and now, they're going to be driving for their own enjoyable pleasure rather than between these other reasons. But anyway, sorry. I didn't mean to interrupt you.
Tyler Suiters
No, that's okay. That responsiveness, though, is very much in your court, from a software point of view, because this is the exceptionally low latency you have.
Mark Douglas
Yeah—
Tyler Suiters
Responsiveness meaning the incoming data and then the reaction of the vehicle.
Mark Douglas
Yeah, let's give you some perspective, too. You know, they say race car drivers react at... I've seen all this data between 350 to 400 milliseconds. And normal schmoes and people like us, maybe we're at half a second or 500 milliseconds. And a lot of times when we try to enter these designs at different stages, we find that the perception element and the planning element are the long pulls in the tent, from a software standpoint. Those need to be sub-25 milliseconds, 50, so when the [inaudible 00:18:52] together, the whole trip should be under 100 or 150 milliseconds, because that comes back to that... You know. So that's the latency perspective that we want to lend to this thing, which is a little bit more low-level, as you've got to process all that in that certain amount of time period. And that's well under what we do, and of course, if that can happen, then the result of a ball rolling out into the street or a young child, god forbid, flying out into the road, you've got a lot more reaction time to do something with that. And hence, stopping distance and a lot of different things like Robert [crosstalk 00:19:27].
Robert Day
Yeah. And one of the challenges there is, we've got all these sensors. You've got HD cameras, you've got lidar, you've got radar. It's pumping all of these pipes into the car. So A, you have to get that data to the compute. Then you have to make sense of it, or you have to process it.
Tyler Suiters
Right.
Robert Day
You have to make sense of it, and then you have to decide what to do, all in that timeframe.
Mark Douglas
Yep.
Robert Day
And just think how much compute's needed to do that.
Mark Douglas
Well, there's one element [crosstalk 00:19:50] There's one element that's receiving all of that.
Robert Day
Yeah, absolutely.
Mark Douglas
And so do you have multiple elements receiving it, and then compare the answer?
Robert Day
Yeah.
Mark Douglas
You know, a lot of people are doing things like that. You know?
Phil Magney
And it has to stitch it all together in real-time and do it 20, 30, 40 times a second. You know?
Mark Douglas
Yeah.
Robert Day
Yeah. So, it's very interesting, because the sensors are not just looking forward. We, as humans, typically look forward until we have to do a maneuver. They're looking all around because they have to get the environment that's all around the car. So, they're looking not just for stuff that the car's going to do, but what's happening around it. And therefore, can sort of take evasive action, if necessarily.
Phil Magney
Yeah, for example like some of the level 2+ features that are being pushed out right now, like in the Tesla vehicles where you can overtake slower vehicles, and so the vehicle will automatically change lanes. And of course, to do that, you need a lot of intelligence. You need to be absolutely certain that there's nobody coming from behind at a high rate of speed. Of course, that's a very camera-centric solution. And they're good at it, I'll give them credit. I mean, they, frankly, are one of the best at automated vehicle technologies in doing with a camera.
Phil Magney
But yeah, the level of intelligence and of course the level of safety just rises considerably once you go to the more advanced levels of function.
Robert Day
Yeah. And that safety has to go all the way through. So, you have to know that the stuff you're getting from the sensors is correct. Okay? You have to process it correctly. You have to make the right decision about what it is correctly, and then make the right decision on what to do correctly.
Mark Douglas
And since it's temporal, everything cascades. Right?
Robert Day
Yeah.
Mark Douglas
So everything is dependent on the other one. So every part or every subsystem has to be at a certain safe level, or verified level. That way, that the data that's coming in makes sense, and then there's also that checker at the end, to make sure that there's something [crosstalk 00:21:42].
Robert Day
So I want to come back to something Phil said, as well, is... Yeah, all of the autonomous prototypes out there right now with self-driving car prototypes, you look in the trunk, and it's all full of compute. And it's literally... So it's not deployable. It's too hot, it's too big, it's too heavy, it's too power... It's too everything.
Tyler Suiters
Is it prototype? Is that the right term, at this stage?
Robert Day
Prototype, I think, is the right term right now.
Mark Douglas
Yeah, absolutely.
Robert Day
Because what this is really doing, is this car is testing the sensor and the software to make sure that they can do the right thing; everything we just talked about. So they're throwing compute at it. It's like, "I don't know how much compute I need. I'm just going to stick a trunk full of compute." And that might sound a little bit flippant, but it's sort of what we see on a day-to-day basis. But that's not deployable, because if I get in a robo-taxi and I go to put my bag in the bag, and there's a ton of compute, and the car's super hot because of all the compute, it's like... No, that's not practical.
Mark Douglas
Yeah.
Tyler Suiters
Yeah.
Robert Day
So what we're looking at now, as an industry, generally, but we, specifically, as Arm, is how can we reduce everything; the size, the power? Keep the performance, reduce the thermals and everything else. And luckily, Arm's really quite good at that, because we're in the small things you carry around in the pocket every day with a lot of compute in it.
Tyler Suiters
Is the process parallel to, or somewhat similar to the evolution EVs, electrical vehicles? In that, when you, Robert, talking about a prototype and you've got compute throughout the whole car, I'm thinking back to the first, maybe not the first, but the evolution of EVs and how you just loaded up a car with as many batteries as it could hold. Right? Because they couldn't get small enough or powerful enough. And you didn't worry about luggage, where people would sit, or how hot it got; it was just about what can we do to generate the electricity.
Robert Day
Yeah, that is an interesting analogy, actually, because—
Phil Magney
Powertrain is probably outside of our scope, right?
Robert Day
Yeah.
Phil Magney
I don't know, maybe Mark, you've had some experience in powertrain, but-
Mark Douglas
No, only in the management of what I see the evolution of where everything is going to be reporting up to a central system.
Phil Magney
Yeah, exactly.
Mark Douglas
And managing it, but that's the software-defined vehicle.
Phil Magney
Exactly.
Mark Douglas
Which is a generation or two above what we're talking about.
Robert Day
But the complexity is so up high in this. You're not just optimizing battery technology; you're optimizing software or you're optimizing where do you stick the compute?
Mark Douglas
Yeah.
Robert Day
There's this whole thing right now, is do you have one big central brain, like a human, that's taking all this input and... I'm waving my hands, there. Taking all this input, and then processing it like a human brain does, or do you actually start putting intelligence towards where the sensors are? And so each sensor becomes intelligent, so the compute just goes, "Oh. I only need to worry about this sensor, because all the rest of them are saying, 'Oh, there's nothing out there, don't worry.'"
Tyler Suiters
Well, where you do all lie on that? [crosstalk 00:24:11].
Phil Magney
There's different schools of thought. I mean, at one end of the spectrum, you say, "Okay, let's take all the raw data we can get."
Tyler Suiters
Yes.
Phil Magney
And then put it in the central compute and try to make sense of it that way. The other school of thought is, let's take intelligent sensors that can produce object data and work with that. And typically, I would say, as far as prototype vehicles, it's a combination of things. I mean-
Robert Day
It's mainly central compute right now. And I think it's because then, the people developing the autonomous system kind of own all of it. It's like, "If... " I'm not necessarily ready to trust that distributed sensor telling me something. Whereas, if I'm just getting feeds that are just raw feeds, I trust that everything I do with my software and my hardware. But I think now, they're looking towards deployment, it's different. It's like, "Okay, I can't just deploy this trunk full of compute. I'm going to have to optimize it."
Mark Douglas
Right. Yeah, that's why the—
Robert Day
—That's what I think, Phil. I mean, just—
Phil Magney
—Yeah, I would completely agree. Yeah.
Mark Douglas
Yeah, that's why software or complimentary software for optimizing the training of the neural nets and the accuracy of them will prevail, I think, in the long run, because then you can put lower quality cameras, you can put three or four cameras in different areas and do different things. And then always select which one, or compare the answers of those things as they come in. I think the one thing that we're seeing folks do is they're putting enough sensors on those vehicles for the... There's two different markets now, too. There's the level 2+ that we were talking about, Phil was talking about. And then there's the other case. And they don't necessarily scale. You couldn't build one and have it go into the full autonomy scale way, or you couldn't do something inexpensive with full autonomy and bring it into the lower end. That's my feeling, there. But I think a lot of times, people will put enough sensors so that way, you can drive the vehicle with half the compute, and so on.
Phil Magney
And some sensors, too, you're getting some innovation... A lot of innovation, actually, in the sensor space, where they're coupling a couple of different sensors in one module. So naturally, they apply some processing and some software there to totally synchronize that and filter out the junk and give you a much better data that you can build into your environmental model.
Robert Day
And I think we're seeing that with some of the level 2 stuff right now, with the cameras and radar being combined [crosstalk 00:26:25] So you get this sense of, "What do we... " Yeah.
Mark Douglas
Yeah, so I'll just say lidar with multiple cameras, and that will come in one particular chassis, one particular sensor. And that can be placed, and so you get a very rich reception, a very believable, you would think, perception result. And that's usually the ID of the vehicle, the tracking of the vehicle, the vector or the direction it's going, the rate. And then there's a prediction element based on what it's doing. So we can see that in a lot of the typical [crosstalk 00:26:57].
Robert Day
And the other thing is, as you distribute, I mean, as you go out towards the sensor edge, as we call it, then you have to get the size down, you have to get the power down, you have to get the cost down. Okay? And so that whole economy of, "Well, I'm moving some of the compute around," but I have to make it a lot smaller or a lot more power efficient, because it's going out to where the sensor is.
Tyler Suiters
So this is just one area of central versus distributed. If we could back out to go macro for a moment, how many different topics of conversation are there with our various schools of thought across the self-driving vehicle right now, about the elements that make it? I mean, almost polar opposites in terms of how you get to the same place; very different directions.
Phil Magney
Well, central domain computing has been a topic... It's been talked about for probably 10, 15 years. The auto industry has moved very, very slow toward that. Right? But they're beginning to see and recognize the benefits of that central compute architecture. The one OEM that really came in and did it totally different, and obviously that's Tesla; that is, in my opinion, that is a proxy for future vehicle architecture, because it is truly a software-defined car, and I think that is honestly the direction that a lot of OEMs are going with respect to their architecture. How long it'll take them to get there, I don't know. Because they're up against 100 years of doing things the old way, and that was a highly distributed system where you've got all this logic distributed and pushed out rather than centrally managed like Tesla does.
Mark Douglas
Yeah. And that's a disruptive OEM. Right?
Phil Magney
Yeah.
Mark Douglas
That's not a traditional Ford, GM kind of local, been around forever and they're evolving. But that, it's very different.
Phil Magney
Yeah.
Mark Douglas
And we have other guys on the scene, Rivian, other folks, as well, that are doing it.
Phil Magney
Sure.
Mark Douglas
But I always feel like every time I go in to talk to somebody, with whatever they're doing, I got to establish a lexicon of some sort. Like, "What do they mean why they say this? What do they... "
Tyler Suiters
It's not common, Mark.
Mark Douglas
No, it's not. Yeah. Nobody... Yeah, nobody shares it, and you don't really understand. And I don't really know that Tesla has a software-defined vehicle. I just know that a lot of people are looking around, they're seeing the compute, they're seeing these extra cycles, they want to add other features to those cycles. And then, of course, there's this... I think we're going to see the same phenomemon we saw in the networking space, which was... When we evolved to software-defined networking, is this consolidation, I don't know if it's in one SOC or multiple SOCs, but I think there's going to be a force of that in what'll start coming out as virtual machines and different things that need to provide protection and all that kind of stuff.
Mark Douglas
So everybody's probably going to evolve in the way where their lineage is; where they're coming from. You know? I don't know that... I mean, maybe there'll be people that, or developers that'll start fresh. You know? But I don't know, I—
Robert Day
Well, so that... Yeah, sorry.
Phil Magney
I was just going to, just quickly, and then I'll let you jump on that one. But I was going to say that one of the things that OEMs really would like to be able to do is this remote feature enablement. So you configure the cars the same, they're built the same to the same specs, the vehicles hit the showrooms, and then they get in the customer hands, and then the feature enablement is done over the air with software. And so you pay the extra $5,000 or whatever they're asking these days for full self-driving, and then they push it out to you, and then you have it.
Tyler Suiters
And that's the aftermarket of the future?
Phil Magney
Well, I don't know if I would label it the aftermarket. That is something, now, that is highly vertically integrated. I mean, you really... Well, I don't know where we'll be 50 years from now or 100 years from now, if the aftermarket is actually going to be able to get down into the ECUs. It's kind of a scary proposition. But no, the ability to enable new features and options remotely via software.
Robert Day
Yeah. So that's a really good point. It's like, "Well, if I need an extra 100 horsepower, oh that's a software tweak, we can do that."
Tyler Suiters
It's a download.
Robert Day
Which is scary.
Mark Douglas
So my brain is thinking, this whole thing where Phil just opened up is like, yeah, we knew it was there, and we didn't really talk about it. There was another elephant in the room. But okay, so I got a whole new set of code going down. I got to bring that in, I got to not let it touch, it's got to stay contained, it's got to be in its own sandbox, I got to boot it up on some additional... This is where, when I'm talking about VMs being available on the SOCs; booted up in there, tested, run some tests, and then say, "Okay, now it's deployable." And then when does that get deployed? Is that when I shut the vehicle down, and then it gets deployed and then fired up and then it's tested throughout that process and then I start driving it? Or, do I... So there's a huge... I see it in my head, that all these... A containment of the domain and then all that stuff.
Mark Douglas
And then, there's folks that, when they drive throughout the road, throughout wherever they're at, with all the senors that are collecting this data, people want that data, and they want to get it up to the cloud and then... So there's both directions. They want to get it up to the cloud and then share it and then enhance other things. There's a lot of other potential. There's law enforcement potential, there's a whole bunch of different things; advertising. And that data's coming back up from the sensors that were on the vehicle that were recording as they drove down the road. So you know what I mean? So it gets kind of mind-boggling.
Phil Magney
And the other elephant in the room is, then, who owns the data?
Mark Douglas
Yeah, exactly. [crosstalk 00:32:33].
Tyler Suiters
Pause there, pause there. That's a separate podcast or four.
Robert Day
Yeah, that's right. So one of the things that's really interesting... Sorry, I'm... That's all right. Sure? Okay... is the difference between ADAS, which is level 2 and a bit of level 2+ and autonomous; the difference is so huge that there's not one company who's really going to be able to do autonomy.
Mark Douglas
That's right. Completely different market.
Robert Day
Absolutely.
Mark Douglas
That has to be understood.
Robert Day
So the traditional automakers, the OEMs and the tier ones can do ADAS. Okay? They can get there. They've proven it. It's in our cars. So we have emergency automatic braking, all that sort of stuff. Autonomous is just too big. And so what's happening now, is we're seeing relationships; we're seeing people starting to work together. And these are people that don't traditionally work together, are now saying, "Yeah, this is a bit too big." So the autonomous technology providers are working with the OEMs. The tier ones are coming in. Everybody's trying to figure out, how do we actually do this and get this to deployment? You've been doing your prototyping. Everybody's saying we're still a way off. When we're actually going to deploy this, how do we get there? And we're starting to see a lot more kind of consortiums being formed, we're seeing a lot more kind of interested in partnering and working together as an industry. And that's a real shift in the [crosstalk 00:33:51].
Phil Magney
Ecosystem management is obviously a key term, now. Everybody is talking about who's within their ecosystem and who are their partners and so forth.
Tyler Suiters
Robert, I don't want to lose track of this point, that Arm has a heavy presence at CES 2020. How much can you tell us about what you all have planned for the show and the meetings and the engagement and what you'll do at the center of the tech universe during the show?
Robert Day
So CES is a very busy show for Arm. If you think about consumer electronics generally, well, that's Arm's sweet spot. So most of what you will see in most of the halls is somehow powered by Arm. When it gets to automotive, it's interesting because the automotive presence at CES has grown dramatically. But it's not always obvious where Arm is in a car. And so what we typically do as Arm, is we kind of produce an automotive kind of guide to what's interesting to see on the show floor with Arm tech in it. So we will have booklets at the show, that you can come and pick one up that will basically give you a guide to all the fun, cool stuff that's actually powered by Arm devices in automotive. And it's amazing, the different types of systems from entertainment to autonomous systems to just everything that's going on that's cool at CES, powered by Arm. Yeah.
Mark Douglas
Completely agree.
Tyler Suiters
What a great launching point for a final lap. All right? I'd like to give each of you one sentence, it can be a run on sentence. And I'm going to give you a timeframe. So let's bring it back to almost where we started here. Phil, for you, one sentence on where the industry will be five years from now.
Phil Magney
Okay, five years from now, I think we will see a lot more on the ADAS side, a lot more of the incremental automation and active and advanced safety features brought out to series production cars, because frankly, in my opinion, you are going to see... This business is not going to change overnight, and you are going to see series production cars sold as they've been for the last dozens of years, for the next 10 or 20 years. It's going to be a while before that starts to fall off.
Phil Magney
So in five years, I would say, yeah, we're going to see a lot more in series production, but then we'll be a lot further ahead in terms of the level 4 stuff, and they'll be some deployments in some parts of the country on a very limited basis.
Tyler Suiters
All right. Mark, Let's double that timeframe.
Mark Douglas
Follow the money.
Tyler Suiters
A decade from now.
Mark Douglas
Oh! Follow the money on any of these questions. Because right now, what I'm seeing is... And I was really excited about... I still am excited. But about a year ago, everybody was doing these things and now people are trying to enhance them. But I'd say between five, ten and beyond, if you look at where the money is, where things are going, it's still in ADAS. It's still level 2, level 3. I don't see a huge mad dash to get to where people were predicting full autonomy would have been in 2025. It's getting bumped out. And I'm hearing numbers like 2030 and so on. And I don't see the investment. I see the investment... And again, there's two markets, here. There's the safety and there's the level 2+ kind of stuff and people creeping into level 3, and people have opinions about that. But then the full autonomy, it will be sandboxed. It'll be somewhere around, with a lot parameters. I just kind of... I don't see a mad dash to that, because I don't see the volumes. Typically don't see the volumes in that, at the traditional level. So I just see our current vehicles getting much smarter as they move along.
Tyler Suiters
All right.
Mark Douglas
And maybe there'll be a point where that chasm will be crossed. I bring that old term up, but that's... I don't know. When you asked Phil that question, I was like, "Well, I'm glad you asked him first." Because then I thought, "Yeah, follow the money." You got to follow the money.
Tyler Suiters
Yeah.
Mark Douglas
Because a lot of this stuff is great, and we can get all this technology and try to ram it into something, but it's how practical is it going to be, and where is it going to result in?
Tyler Suiters
Right, right. So Robert, final to you. 25 years from now. It's 2035. Oh, no. 2045.
Robert Day
2045.
Tyler Suiters
Working on my math very quickly, now.
Robert Day
Yeah, it's good.
Tyler Suiters
2045.
Mark Douglas
I'll have a walker, then. I don't know about you. I may not even be able [crosstalk 00:38:07].
Robert Day
Well, yes.
Tyler Suiters
But you can go anywhere you want, Mark. [crosstalk 00:38:09].
Robert Day
That's right. So an autonomous car will pick Mark up every day, that's for sure.
Mark Douglas
If he's still alive.
Robert Day
That's right, if he's still alive. So I think what will have happened by then is car ownership will have changed. We will not own our own cars anymore. Or at least, if we do, they'll be hobby cars. Okay? Ride sharing will be an interesting thing. Whether it's dominated by the current ride sharing companies or whether we all do our own kind of micro-ride sharing in that I'm going to buy a car, but then I'm going to share it with everyone else. Or, I'm just going to share somebody else's car. So basically, our whole model for car ownership and how we use cars, whether we're self-driving or not self-driving, is going to change. But I think there will be... People will actually start to see the benefits of having cars making decisions versus humans on the road. And I think, at that point, 25 years from now, we could well see more self-driving cars than driven cars on the road.
Tyler Suiters
And much safer roads, highways.
Robert Day
Oh, god. Yes.
Tyler Suiters
Streets.
Robert Day
Yes. Yeah, you can comb your hair and eat without crashing.
Tyler Suiters
Legally, yes.
Robert Day
Yes.
Tyler Suiters
What a conversation. Robert Day is Director of Automotive Solutions and Platforms at Arm. Mark Douglas is Field Software Solutions Architect at NXP, and Phil Magney is Founder and Principal Advisor at VSI Labs. Gentlemen, I feel like I'm flipping on the lights right before closing time and killing a great conversation. But what a blast today. Thank you all for the time.
Robert Day
Thank you.
Phil Magney
It's our pleasure.
Mark Douglas
Our pleasure, yeah.
Phil Magney
Thank you.
Mark Douglas
Absolutely.
Tyler Suiters
All right, coming up on our next edition of CES Tech Talk, a very hot topic here in Washington for various reasons. But really, one that's happening around the world: a look at cryptocurrency. That is on our next edition of CES Tech Talk.
Tyler Suiters
We are here to help you get CES Ready, so do yourself a favor—subscribe to this CES Tech Talk podcast. That way, you won't miss a single episode as you're getting ready for the next show. Speaking of, CES 2020 is January 7th through the 10th in Las Vegas. The information you need to prep yourself and your trip is at ces.tech.
Tyler Suiters
As always, none of this is even remotely possible without the true stars of our podcast, Executive Producer Tina Anthony and our Senior Studio Engineer, John Lindsey. Y'all are the best in the business.
Tyler Suiters
I'm Tyler Suiters. Let's talk tech again soon.
