Lenny's Podcast: Product | Career | Growth
00:06:54 · Tulsee Doshi
[00:06:54] S0: First of all, hopefully by the time this conversation goes out, you will be using Gemini 4 Argonne, which is exciting. And so I really do hope we're having a conversation about a frontier AI model that folks are excited to use. And I think Gemini 4 speaks a little bit to our strategy, right? Because it really is our push to continue to build a frontier lab, a frontier set of models, a frontier set of capabilities on our path. I think if you look at Google's strategy, it really is full stack, right? So one of the things that I think is powerful about Google is all the way from hardware to end products that serve billions of users. I think the stat is like we have 14 products that are above a billion users. I should check that number, but I think that's the stat. And so if you think about that, we actually go all the way from the hardware to then the modeling to then this kind of end user application. And our goal ultimately is to not just build amazing frontier experiences, which is definitely a part of the goal, but it is also to build models across this Pareto curve of cost to quality that can serve different users at different points in their lifestyle, whether that be a search user who is coming for Information, right? Whether it is an engineer who is trying to build the next amazing thing, whether it is for a cyber defender who is trying to protect a critical piece of infrastructure. I think our goal is to make sure that we have models that serve that range of use cases and that we're actually building with feedback from these users.
00:08:30 · Lenny Rachitsky, Tulsee Doshi
[00:08:30] S1: So what I'm hearing is there's so many vectors where Google can win. There's the most efficient kind of Pareto Frontier. For token costs, there's the smartest model. There's also just making your products better. For example, AI mode, AI overviews. And so it's not that you need to be the best at the most advanced AI model that people can use, like Chatbot, basically. You can win in all these other areas. But it sounds like you're also trying to win and have the most state-of-the-art model [00:08:58] S0: I actually think we do want to win the frontier conversation, right? I don't think we should say that Google is not participating in the frontier. And I think we are building better and better models. And I think Gemini for Argonne definitely is at the frontier in the place that we are at. But I think one thing that is amazing is being able to build a breadth of experiences and build a diversity of experiences for users. And I think for Google, Our goal ultimately is to build amazing experiences that actually make people's lives better, right? And that is, I think, ultimately the goal, whatever part of the frontier you're building for. [00:09:42] S1: Google products definitely make my life better. It's just like amazing how much of your life is within a Google ecosystem, like not even the AI stuff, but just Google Calendar, Gmail, all these things, you're just like Google Docs. So it's happening. [00:09:55] S0: There you go.
00:09:55 · Tulsee Doshi, Lenny Rachitsky
[00:09:55] S0: There you go. [00:09:56] S1: There we go. I want to talk about one other broad AI question that I'm going to dig into more product specific stuff. A big topic of conversation these days is AI safety and this kind of fear that AI is going to harm humans. It's going to cause damage as it gets smarter and smarter. You have a really interesting lens on this because you led product for the responsible AI team at Google for six years. This is what you did right before this role? And so you've been thinking about this for a long time, a lot longer than most people. I'm curious just to get your take on this debate. Just, I don't know, are you optimistic we will figure this out? What do you think people maybe don't understand about this debate around AI safety?
00:10:36 · Tulsee Doshi
[00:10:36] S0: Our safety teams here are incredible. And it's been incredible to see the intentionality that they've been putting in all the way from, you know, Initial trainings of the model to how we deploy them. And I think safety is embedded in that process. Right. And we need to be. I think the part that is important, actually, as we talk about how do you deploy models safely is actually thinking through what are the right sort of mitigations and structures around these models. Right. So, for example, one of the things that's really important is thinking about monitoring. Right. And so for us actually deploying monitors that can monitor the model's chain of thought and actually identify whether the model is misaligned with its directed actions or whether it is aligned and then being able to intervene appropriately. That's actually the kind of mitigation that gives us more confidence in deploying these models, but also allows us to better understand where we might see misalignment and better actually improve the models over time. And so I think Things like monitoring give me more confidence in how we're putting our models out. I think the other thing that we don't talk about enough when we talk about this debate is also the role that AI can play in hardening our systems, right? And so one of the things that we're doing with Gemini 4 is actively partnering with a number of cyber defenders, including actually just even partnering with Wiz, that is now a Google company, to really think through How can we actually identify and patch vulnerabilities in systems across the globe? Right. We actually have this interesting case study. Wiz has this scan for good program where they're actually like actively working with systems across the world to identify, you know, potential issues. And they worked with a hospital system to identify a vulnerability and then patch it. And it's that kind of thing that I think if we can actually leverage these tools to do more of, we can actually make sure that our infrastructure and our systems are robust. Right. Which is also, I think, really critical as this whole space grows.
00:12:34 · Lenny Rachitsky, Tulsee Doshi
[00:12:34] S1: There's actually a conversation on All In this last week about this whole issue that now that AI is getting really good at cyber, people are going to be investing a lot more in cyber defense and AI for protecting. And so there's going to be a lot of money being spent on protecting their systems from this ever smarter AI. So it's really cool that you guys are investing there. So I guess just coming back to the question of, do you feel like we'll figure out this concern and this issue we feel, a lot of people feel that AI is going to Take over the world, cause a lot of damage. Do you feel like this is a solvable problem? Are you optimistic or is it something you think a lot about and are kind of worried about? [00:13:12] S0: Maybe the answer to your question is both in the sense that I think we need to be consistently thinking about it. Right. I think the way to optimism here, for lack of a better way of putting it, like the path to optimism is through thinking about it and being like very careful and very thoughtful and very intentional on the path to get there. And so that is like my source of optimism comes more from Do I feel like we have people who are being intentional and thoughtful about what the safety story looks like? And I think that is the critical threat is you need a number of people to be worried and to be having conversations and to be actually interrogating how we're building our models. And that's the step by step process we have to take.
00:13:12 · Tulsee Doshi, Lenny Rachitsky
[00:13:12] S0: Maybe the answer to your question is both in the sense that I think we need to be consistently thinking about it. Right. I think the way to optimism here, for lack of a better way of putting it, like the path to optimism is through thinking about it and being like very careful and very thoughtful and very intentional on the path to get there. And so that is like my source of optimism comes more from Do I feel like we have people who are being intentional and thoughtful about what the safety story looks like? And I think that is the critical threat is you need a number of people to be worried and to be having conversations and to be actually interrogating how we're building our models. And that's the step by step process we have to take. [00:13:50] S1: Awesome. And what's interesting is I don't think I've read a story of Google's models hacking into anyone, which I think shows that this is something you all have been thinking about for a long time. I don't know, for better or worse. But it feels like it's working, whatever y'all are doing. [00:14:05] S0: We like stories about fixing vulnerabilities. That's a good place for us to be.