00:16:37 · Ruchi Sanghvi, Dmitri Dolgov, Aditya Agarwal
[00:16:37] S1: Do they all run on the car itself? Like, does the simulator need to run on the car, or can it run on the cloud? [00:16:43] S0: Oh, no. Absolutely not. So the only thing that runs on the car is the driver, And the other two parts are in service of training, evaluating, and validating the driver. [00:16:52] S2: Do you think that the architecture that you have described is an emergent one based on how you have seen almost the path dependence of the learnings over the last seventeen years of building this? Or would you almost derive this kind of three models, kind of like the simulator, the driver, the critic, does that feel like the right shape of architecture for real world, low latency, high feedback loops for a variety of different kind of devices and so on?
00:17:22 · Dmitri Dolgov
[00:17:22] S0: Yeah, think so. With some caveats. On and I think the I guess I may jump to that. I think the key distinction and the key aspect of the problem, whether you need all three, will be the level of complexity of the task. And the key distinction will be whether you can get by with kind of open loop imitation and open loop evaluation and validation, which for some applications is sufficient, not in our case. And if it's not sufficient, then you do need the entire ecosystem. So the way in our space, I think you'll see the same property merge in other fully autonomous applications of AI in the physical world where you cannot sufficiently control the environment or make the problem simpler. If open loop imitation and open loop evaluation and validation isn't sufficient, that means that closed loop is required for validation, as in the case of our product, and it is incredibly valuable for training. Right? There's some things you can do in closed loop training that you cannot do in open loop. Then that leads you to needing a high fidelity realistic simulator. Right? Number one. And number two, if you are in... Playing in this closed loop training ecosystem, then the separation between actor and a critic, as often happens in kind of reinforcement learning frameworks, is a very natural one. So then if you are in this closed loop as required, then I think you very naturally end up with a trifecta.
00:19:05 · Aditya Agarwal, Dmitri Dolgov
[00:19:05] S2: You've often said that the hard part of driving isn't just the perception. It's the fact that there are multi agents kind of like each of them kind of playing their own part. You're having to essentially model out not just your own behavior but how the world is going to react to some of your own kind of actions. And then you pair that with kind of the closed loop RL which is what gets you to superhuman safety. But how do you think about the closed loop RL in a world where your bad trajectories lead to very bad outcomes? [00:19:34] S0: First of all, you don't do training in the real world. [00:19:37] S2: You have [00:19:38] S0: to say that. You can't learn the real world is for data collection, for grounding. All of the training and all of the closed loop training especially happens in the simulator. But, of course, you cannot build a realistic high fidelity simulator in a vacuum. It's an accelerator, but it still has to be grounded. And this is where the tight loop of building the driver, the simulator, the evaluation machinery, and kind of letting them evolve together and be grounded and validated in real world data with real deployments is what's required.
00:19:38 · Dmitri Dolgov, Aditya Agarwal
[00:19:38] S0: to say that. You can't learn the real world is for data collection, for grounding. All of the training and all of the closed loop training especially happens in the simulator. But, of course, you cannot build a realistic high fidelity simulator in a vacuum. It's an accelerator, but it still has to be grounded. And this is where the tight loop of building the driver, the simulator, the evaluation machinery, and kind of letting them evolve together and be grounded and validated in real world data with real deployments is what's required. [00:20:17] S2: We've talked publicly about how the fifth generation driver was where you kind of made a big jump, right? Like to kind of like being a I guess you describe it as AI as kind of the more of the backbone. What kind of motivated that change? Like it sounds as though that was in some ways a fairly big architectural kind of like difference as you kind of made that jump. Like why what ended up kind of motivating that?
00:20:40 · Dmitri Dolgov
[00:20:40] S0: Well, it was a natural evolution of the state of the art of the tech. I think around that time, I think it became pretty clear that we wanted to kind of bet on this paradigm of building high capacity outboard models and then distill them into the students, Waymo Driver that runs in the car being the student. You just get better scaling loss that way. And it's not specific to autonomous vehicles. It's just kind of a fairly known property that you start with high capacity models, and then you distill. You get better scaling loss than just from working on the small capacity models directly. And so in order to do that, you want the... Your, you know, your your driver to be an effective student. So that means it needs to have kind of maybe, you know, have a strong ML backbone. Otherwise, you just can't, you know, learn from the teacher so well. Right? And also at that time, you know, we've gone through several big waves of innovation in this project. Right? We've, you know, we've seen ConvNets, Then, you know, we've seen transformers. You know, now there's VLMs, VLAs. Right? So it just seemed like a very natural progression. You mentioned end to end as, you know, one of the kind of design dimensions in this whole space. This was also all of that was coming together.
00:22:00 · Ruchi Sanghvi
[00:22:00] S1: You've been through multiple innovations in the course of this project, but it feels like these innovations were so significant that you would have to rewrite large parts of the system to take into account these innovations? I mean, do you do that? Like, how do you rewrite a system, especially when it already works?