Lenny's Podcast: Product | Career | Growth
00:14:05 · Tulsee Doshi, Lenny Rachitsky
[00:14:05] S0: We like stories about fixing vulnerabilities. That's a good place for us to be. [00:14:09] S1: Okay. So one of the interesting things that's happening kind of from a product perspective with AI is Products are getting very complicated. For example, Google gets a lot of flack for it being hard to find all the magical things you all do. Like a lot of people don't know AI Studio even exists. A lot of people can't find VIA or Genie and all these things. Similarly, Codex, ChatGPT recently launched Dots and now you go to the interface of Codex and it's like tabs and dropdowns and Dots and it's just like out of control. And Claude Code is the same thing. It's just getting very complicated. That code work and then they drop code work and And interestingly, I'm in this WhatsApp group with a bunch of PMs, and there's a PM from one of the labs. [00:14:51] S0: Well, I won't be in this WhatsApp group. Sorry. Okay. [00:14:53] S1: Let's talk. Let's talk. And he showed that there's this kind of urgency to get frontier power of AI out to users as fast as possible, partly to stay competitive, partly because they don't know exactly what it's used for yet. So let's get it to people, figure out what they're using for it, and then make the UX better. So there's this kind of intentional, let's just get it out there, figure it out later how to make it a great UX. Thoughts on this kind of balance of making the product really easy to use versus just getting things out as quickly as possible. And is that something you deal with?
00:15:26 · Tulsee Doshi
[00:15:26] S0: Two thoughts, maybe, maybe three. The first one I'll say is I do think we do need to be playing with new paradigms and new UX approaches, if you will, to how people will engage with models. Right. So you mentioned, for example, Genie or Veo, which is which we're now building Omni. Like if you look at, for example, Flow. Flow is a product that we've been designing, thinking about video editing in generation, music editing in generation. And so I think we are trying to play with like NotebookLM is a good example of a different form factor. I think our labs teams are actively thinking about like, how do we try different form factors in different UX for different individuals to engage with these models? And so I do think we need to keep innovating and playing around with what UX looks like. I think at the same time, Your point is also right, which is like, I think depending on who your user is, you want a different form factor too, right? So like, I think, for example, with AI mode, we're talking about billions of users coming to search, asking their questions. I think you do need to have a certain UX there that is accessible to every user, right? At the same time, on the other end of the spectrum, when you have something like anti-gravity, you are trying to build for a power user. And you are trying to build for users who know how to use slash commands and can actively engage across both the terminal and the IDE. And you want to be able to support kind of a wide range of what a user can do. And so I think depending on who your user is, you have a certain level of flexibility with how you think about the UX, right?
00:16:58 · Lenny Rachitsky, Tulsee Doshi
[00:16:58] S1: That makes sense. Basically, there's different services for different levels of how radical you can change things. [00:17:03] S0: Yeah. And I think the other part, too, is like, I think it's also important to think about the model as having a UX also. Right. So one of the things I think about a lot is like a PM for the model is like, what is the UX of actually using the model? Right. So how transparent is the model? How well does it communicate with you? What questions does it come back and ask you? Right. These are all actually this is all UX in its own way. And so I think it's also really interesting to think about, like, how do we bring conversations about UX into the actual model development process itself? [00:17:31] S1: That is really interesting. Basically, the personality of the model is the UX of the model. How do you, as a PM, inform that work with the researchers? Is it like, here's feedback on the way it's talking, and it should be a little more verbose here, and here's ways it could format things differently?
00:17:31 · Lenny Rachitsky, Tulsee Doshi
[00:17:31] S1: That is really interesting. Basically, the personality of the model is the UX of the model. How do you, as a PM, inform that work with the researchers? Is it like, here's feedback on the way it's talking, and it should be a little more verbose here, and here's ways it could format things differently? [00:17:46] S0: I think the role of the PM here is actually really awesome because you are actually getting to shape what is the direction of where we want to go with the model's behavior, right? And so we have a team that works on model behavior here. And a lot of what we've been thinking about together is, A, looking at customer feedback and user feedback for how they engage. But also thinking through what does good look like for Gemini specifically, right? And how do we develop kind of a spec, if you will, for what does that look like? And then being able to translate that to rubrics, right, that we can then evaluate the model on to be able to translate that into data that we can actually leverage for the model to be able to translate that into our reward modeling. And so it's been actually like a really nice collaboration between PM and the research team. [00:18:30] S1: Something I hear a lot is that even if we stopped development on the model, there's so much it can do that we are not yet doing. There's this overhang. Is that something you think a lot about? I feel like that's a big part of this job is exposing the powers of the AI model to users.
00:18:30 · Lenny Rachitsky, Tulsee Doshi
[00:18:30] S1: Something I hear a lot is that even if we stopped development on the model, there's so much it can do that we are not yet doing. There's this overhang. Is that something you think a lot about? I feel like that's a big part of this job is exposing the powers of the AI model to users. [00:18:46] S0: Yeah, and figuring out what is new, right? And where is the next set of exciting? Where is the next frontier? What are the capabilities that we're trying to invest in and build? I think is a fun activity here also. And I think a lot of this actually is really working in hand in hand with our internal customers. If you think about like search or the Gemini app or, you know, YouTube or labs, but also working with external customers to really understand where are they going? What are their needs? What would meaningfully change their experience? And how do we actually build that flywheel of feedback too?
00:18:46 · Tulsee Doshi, Lenny Rachitsky
[00:18:46] S0: Yeah, and figuring out what is new, right? And where is the next set of exciting? Where is the next frontier? What are the capabilities that we're trying to invest in and build? I think is a fun activity here also. And I think a lot of this actually is really working in hand in hand with our internal customers. If you think about like search or the Gemini app or, you know, YouTube or labs, but also working with external customers to really understand where are they going? What are their needs? What would meaningfully change their experience? And how do we actually build that flywheel of feedback too? [00:19:21] S1: Along these lines, one of the other really interesting trends I've been watching on Twitter in particular is people realizing the value of product managers again. We went through this phase of just like, we don't need PMs anymore. Yeah, exactly. It's just like, okay, we can build anything. We don't need PMs. Let's just build it all. PMs are so back. And the reason just to kind of add a little context there is now that it's so easy to build, people are just building all this nonsense and things that don't work. People, things people don't want to use. And people are realizing there's a very specific skill to figure out what is worth building, uh, What problems people have that are worth solving, what a great looks like, what good looks like once you're done, aligning people behind one cohesive vision. Is that something you're noticing? Thoughts on that kind of PM ascendance that has started to happen again?