00:37:26 · Ruchi Sanghvi
[00:37:26] S1: So in your case, like, you know, let's say someone was exploring autonomous systems. You mentioned you did a lot of research before you even thought of building a product. The demo only took about eighteen months, but the actual product took about fifteen years. So for everyone in this room, where the demo actually works, what are the signals you need in maybe a year or two that tell you you're on a fifteen year old fifteen year problem solving trajectory or you're working on a system that you could produce a workable product in the next three years? And I get the fully autonomous versus not is a good, like, rubric, but are there other things?
00:38:06 · Dmitri Dolgov
[00:38:06] S0: Let's say you're working on a problem. You said you have three or fifteen years. Let's say you've been working on it maybe for a year or two, and you kind of want to see if you're a year away or ten plus years away. One thing, kind of one pattern that I've seen over and over again is do you have convergence of discovery? Right? Is the problems that you are seeing today, how different are they from the problems you saw six months ago? Do you feel like as you're pushing on the system, does it feel like you have stability and you're just kind of iteratively improving your metrics in the same areas, or does it feel like every six months, the landscape is shifting from under you and the problems of today are very different from what you saw a while ago? So if you don't have that stability, then that's often kind of a good indicator that you've got some ways to go. The second thing I would mention is judging how close you are to a real deployment, a real product is, again, I go back to metrics and evaluation, and do you have a scorecard? If you think you are, you know, six month or a year away from deployment, do you have a scorecard that if it were green today would allow you to deploy with confidence? If you don't have one, you're kind of saving that for later, often that turns out to be more difficult than you expected and there's a long distance to go. But if you already have one and you're seeing stability in your discovery and stability in your metrics and it's just a matter of closing the gap, that is a really good indicator that you are getting pretty close.
00:39:46 · Aditya Agarwal, Dmitri Dolgov
[00:39:46] S2: Those are both really good, especially tying that back to what you said in the beginning, which was when you started at Waymo, you weren't trying to build a product. You were just like, we were trying to figure out feasibility and we set out some very concrete milestones and goals. And I'm assuming you must have seen that you were able to make progress without it kind of shifting from underneath you to kind of give you confidence to keep on laddering up from there. [00:40:12] S0: We were not in that stage. We weren't in end to end. As I said, this was about, these goals that we picked, this was not in service of building a product. This was in service of us learning. [00:40:25] S2: This learning. [00:40:25] S0: Okay. And I guess this is where I would make a clear distinction. Right? In some ways, if you take my own advice, you're working a hard problem, you start for the fundamentals before you start specializing and thinking about the product or an application. That was not the very first thing we did. But that thing was not in service of... That integration of all the different components at the very, very early stage was not because we thought we're gonna be that close and we're gonna tweak it and have a product in six months. It was because we needed to bring everything together to really deeply understand the complexity of the task ahead of us. Then once we achieved that, we kind of went back to the lab and started looking at the fundamentals. Got it.
00:41:05 · Aditya Agarwal
[00:41:05] S2: One of the cool things that happens in Silicon Valley is that as a new architecture, as a new almost modality of company comes about, that informationknowledge tends to diffuse out into a bunch of different organizations, like whether it's Google Brain or OpenAI that figures out how to build transformers, now we have 10 companies that know how to kind of do that if not more. And so do you think it's a reasonable bet that a few years from now a lot of what the shape and structure of how to create a company that has like a rigorous infrastructure for evals, for testing, for rolling out kind of things that operate in the real world, do you think we will know how to do this as an industry, like call it five years from now, in a way that perhaps entrepreneurs here can maybe take that bet that that is much more, that will become a thing that we know how to do as an industry? Because I would argue that there are not that many. I mean, there are some companies doing robots kind of in the real world, but nowhere close to the scale of like Waymo today. And so is that a reasonable hypothesisbet in the coming kind of, call it, three to five years?
00:42:15 · Dmitri Dolgov
[00:42:15] S0: Look, the way things are going, the AI nowadays, I want to be careful. Not to predict what's gonna happen in five years. As if you asked me this question five years ago, I think I would not have gotten it right. Fair. But I think you're absolutely right that the tools of today and the rate at which they're proving are so powerful that we will be able to share some of those fundamentals. Right? And what you can do with AI agents nowadays and how quickly you can, yeah, kind of give you that foundation is absolutely amazing and mind blowing. So I think it will but my intuition is that it will just kind of how we spoke about the complexity curve as a function of effort in our domain, I think you will see that some of the easier applications and, you know, some of the easier products to deploy, there's gonna be an amazing proliferation of those. I think you will see magic. I think you will see, you know, like, there's, you know, things that, you know, in the past would take, you know, a large company a year. Now you'd be able to do with a team of five people in a month. Right? So something like that. There's some things that I would not expect to fully commoditize, and I think it's, you know, the things where the bar is just excruciatingly high, right, when we're talking about, you know, something like an R domain where the cost, you know, it's multiple components that have to come together. Right? It's the reliability of the vehicle platform. It's the reliability and the performance of all the different components. Right? You have to think about, you know, your power domain, about redundancy and sensing, like everything. The performance of the software in that extremely high number of nines, that, you know, there's a lot of people and a lot of great companies working on this problem. So I'm excited about the progress we're gonna make there. But I think there's naturally is gonna be some applications and some domains that emerge where it's a combination of very multiple multidisciplinary things that have to come together, where the cost, like, the the margin for error is so extremely slim that I don't think that will be fully commoditized.
00:44:30 · Ruchi Sanghvi, Dmitri Dolgov, Aditya Agarwal
[00:44:30] S1: I think you've already answered this question, but it's crushing my dreams a little bit. Does the Waymo Driver ever generalize? Like, is there some sort of general physical agent hiding in your foundation model that can have different embodiments, not in the form of a car? [00:44:47] S0: Yeah. Well, there's... Does the Waymo Driver generalizes when we talk about generalizability... [00:44:53] S2: How to build robots? [00:44:55] S1: Or, like, delivery bots? Or, like, know... [00:44:57] S0: Are we still on record? [00:44:58] S2: Not on this.