Lenny's Podcast: Product | Career | Growth
00:17:31 · Lenny Rachitsky, Tulsee Doshi
[00:17:31] S1: That is really interesting. Basically, the personality of the model is the UX of the model. How do you, as a PM, inform that work with the researchers? Is it like, here's feedback on the way it's talking, and it should be a little more verbose here, and here's ways it could format things differently? [00:17:46] S0: I think the role of the PM here is actually really awesome because you are actually getting to shape what is the direction of where we want to go with the model's behavior, right? And so we have a team that works on model behavior here. And a lot of what we've been thinking about together is, A, looking at customer feedback and user feedback for how they engage. But also thinking through what does good look like for Gemini specifically, right? And how do we develop kind of a spec, if you will, for what does that look like? And then being able to translate that to rubrics, right, that we can then evaluate the model on to be able to translate that into data that we can actually leverage for the model to be able to translate that into our reward modeling. And so it's been actually like a really nice collaboration between PM and the research team. [00:18:30] S1: Something I hear a lot is that even if we stopped development on the model, there's so much it can do that we are not yet doing. There's this overhang. Is that something you think a lot about? I feel like that's a big part of this job is exposing the powers of the AI model to users.
00:18:30 · Lenny Rachitsky, Tulsee Doshi
[00:18:30] S1: Something I hear a lot is that even if we stopped development on the model, there's so much it can do that we are not yet doing. There's this overhang. Is that something you think a lot about? I feel like that's a big part of this job is exposing the powers of the AI model to users. [00:18:46] S0: Yeah, and figuring out what is new, right? And where is the next set of exciting? Where is the next frontier? What are the capabilities that we're trying to invest in and build? I think is a fun activity here also. And I think a lot of this actually is really working in hand in hand with our internal customers. If you think about like search or the Gemini app or, you know, YouTube or labs, but also working with external customers to really understand where are they going? What are their needs? What would meaningfully change their experience? And how do we actually build that flywheel of feedback too?
00:18:46 · Tulsee Doshi, Lenny Rachitsky
[00:18:46] S0: Yeah, and figuring out what is new, right? And where is the next set of exciting? Where is the next frontier? What are the capabilities that we're trying to invest in and build? I think is a fun activity here also. And I think a lot of this actually is really working in hand in hand with our internal customers. If you think about like search or the Gemini app or, you know, YouTube or labs, but also working with external customers to really understand where are they going? What are their needs? What would meaningfully change their experience? And how do we actually build that flywheel of feedback too? [00:19:21] S1: Along these lines, one of the other really interesting trends I've been watching on Twitter in particular is people realizing the value of product managers again. We went through this phase of just like, we don't need PMs anymore. Yeah, exactly. It's just like, okay, we can build anything. We don't need PMs. Let's just build it all. PMs are so back. And the reason just to kind of add a little context there is now that it's so easy to build, people are just building all this nonsense and things that don't work. People, things people don't want to use. And people are realizing there's a very specific skill to figure out what is worth building, uh, What problems people have that are worth solving, what a great looks like, what good looks like once you're done, aligning people behind one cohesive vision. Is that something you're noticing? Thoughts on that kind of PM ascendance that has started to happen again?
00:20:08 · Tulsee Doshi
[00:20:08] S0: I like the phrase PM ascendance. I'm going to use that. Okay. Yeah, I think, I mean, I think you actually said it very well, right? Which is, I think, look, I'm obviously biased because I am a PM. So I would like to believe that, you know, PM ascendance is here and we're back. But I actually think, you know, exactly what you said, which is when you can build anything and you can build anything so easily, building something great actually is now a unique, Value add, right? And taste is now really important. Someone who can come in and actually really pull in what is the signal from the noise? Right. And what is that insight that will actually unlock a great experience? And that is truly what a good PM does, right, is bring that insight and help unlock. I think the other part that you noted that I particularly feel in a place like Google, where we are building a number of products is, again, when this is so easy, someone who can bring people together and drive consensus and turn chaos into order. I think becomes actually really important because in the absence of that and the ability to build as quickly as we now can, speed can basically introduce chaos if you're not actually able to kind of bring clarity to that and bring structure to that.
00:21:25 · Lenny Rachitsky
[00:21:25] S1: So on that, as you're hiring PMs these days, is there anything you... Look for more and more kind of a skill that might be trending up in the PMs that you hire that you think is really important to be really successful or just broadly skills that you find are really important to be successful in this role of being a research PM.
00:21:44 · Tulsee Doshi
[00:21:44] S0: Yeah, I mean, I think more and more so we are looking for PMs who, you For lack of a better way of putting it, understand the vibes of our models or other models. I want PMs who have an opinion, who are playing with these models every day, who have their own use cases for how they judge whether a model is really good, who play with and stress test these capabilities themselves, who are capable of diving in and looking at data and getting scrappy and rolling up their sleeves. And so I think it's both PMs who are trying to build that intuition and As well as who are technical enough in the sense of being willing to kind of like, like I said, roll up sleeves, go in, really understand the data, build an eval, right? Actually sit down and pull those pieces together. I think that's really important. The other thing I'll say is I think PMs who are agile and adaptive are like also really important. I think the space is changing every two weeks. A roadmap you wrote two months ago. You're going to be looking at it again and rewriting it. Right. And so I think PMs who are constantly keeping an eye out on the environment, on the market landscape and actually consistently reinterrogating, like, what are we doing and why are we doing it? As opposed to assuming that there was a fixed kind of plan of action.