The Information's TITV
00:34:41 · Andrew Ambrosino
[00:34:41] S5: Yeah. Yeah. The job has changed a lot. Right? And and I think as we've looked at the... You know, we started the... About a year ago, we started building the Codex app, the original version of this, and the models were very, very different at the time. And so we've kinda seen some of these cycles of how software gets developed and how the models are are, you know, bad at something until they're not. And I think, you know, if you if you look at things like plan mode and goal mode and these kind of, you know, featurettes that we've launched, for software developers, some of them are really meant to be temporary. Some of them are meant to say, hey. You know, in April, the model is good at a lot of this stuff, but it it doesn't keep itself going for a long time. Right? So we introduced goal mode, and goal mode is kind of a more deterministic, more structured feature that says, hey. You know, we're gonna we're gonna prompt the model to keep running until it hits this goal. Right? With newer models, you know, with Astra Right. Goal mode's almost not needed anymore. Right? Because it can keep going. But it's kind of always these arcs of of of things where we're like, hey. We're gonna introduce this, like, little helper here in the software. And at some point, we'll be able to retire it because the model will be able to do that kind of natively. Right? I think where we where we still see the the model's struggle overall across the industry is just, like, what does it mean to maintain a product, in a large product and to take in feedback from everybody and to make decisions about what features to prioritize and and what not to and how to design them. And even though the models are writing most of our code, still, you know, this this this aspect of things is is kind of the challenging part.
00:36:19 · Akash Pasricha, Andrew Ambrosino
[00:36:19] S0: This So is kind of interesting. So so in in your capacity, part of developing the product is sometimes maybe compensating for an area where the model might be trying to make a leap, but it... Maybe it's, you know, it's it's not quite there yet, and so you have to sort of work your way around and introduce something temporary to sort of compensate for that. That's that's kind of an interesting way of doing product development. [00:36:44] S5: It is. It is. And we... The most important thing that we do is get out of the way of the models. Right. And make sure that we are developing software that puts everything that the model can do in the hands of as many people as possible in a way that's as clear as possible. Sometimes this means adding affordances for how to use the model. Right? Adding suggestions, adding proactivity, things like this. Right? Sometimes it means, you know, stripping things out when the model isn't needed anymore. There's this kind of classic thing that, like, model over model. The skills need to change. Right? Because you, you know, you make skills for one and it doesn't work for the other. But, yeah, it's a huge part of the job.
00:36:44 · Andrew Ambrosino, Akash Pasricha
[00:36:44] S5: It is. It is. And we... The most important thing that we do is get out of the way of the models. Right. And make sure that we are developing software that puts everything that the model can do in the hands of as many people as possible in a way that's as clear as possible. Sometimes this means adding affordances for how to use the model. Right? Adding suggestions, adding proactivity, things like this. Right? Sometimes it means, you know, stripping things out when the model isn't needed anymore. There's this kind of classic thing that, like, model over model. The skills need to change. Right? Because you, you know, you make skills for one and it doesn't work for the other. But, yeah, it's a huge part of the job. [00:37:20] S0: So in this world, Andrew, where, I mean, so many of these frontier models are are getting so good, and the competition competition for a platform like Codex is is growing by the week, by the month. How do you think about differentiating yourself and and staying differentiated in this world where, you know, people talk about the models being commoditized? Fine. We can have that discussion, really. I mean, this is a... The chat interface, all these different, platforms use. We... You know, how do you think about the nuances that you want to be ahead in the trade offs on the product level?
00:37:56 · Andrew Ambrosino, Akash Pasricha
[00:37:56] S5: Yeah. Yeah. It's a great question. And it's one we think about a lot. We we are very focused on the frontier, and that increasingly means efficiency. Right? So the most powerful models we can possibly make, most efficient token wise and cost wise is where we wanna be. And then on the product side, you know, model to model, I think one of the great strengths that we have internally is that we understand how the model's trained. We can set up flywheels. Right? We can say, hey, the model's gonna be really good at this on its next release, so let's build this feature to really highlight that, you know, which we've done, I think, a few times. But it's it's it's con... It's constant moving. It's it's never something where we say, hey, mission accomplished. We've we've solved this shape of product. That will never be the case with this. Right? Yes, it's a chat interface, but, you know, with with, GPT six, with Intelligent UI, for example, right, now the chat comes back with little widgets. When you ask for, hey. Where's my flight? Right? It might come back with a widget or something like that. So it's it's constantly changing. [00:39:02] S0: And when you are studying customers using Codex and and other products... I mean, let... Let's just stick with Codex for for a moment. There's so much included in there. How is the way that people are using Codex? How has that changed today compared to even six months ago from what you're observing?
00:39:02 · Akash Pasricha, Andrew Ambrosino
[00:39:02] S0: And when you are studying customers using Codex and and other products... I mean, let... Let's just stick with Codex for for a moment. There's so much included in there. How is the way that people are using Codex? How has that changed today compared to even six months ago from what you're observing? [00:39:23] S5: I mean, people use it for absolutely everything now. I mean, it's... Like, I use it to source feedback even. You know? I... The other day, I tweeted something along the lines of, hey. We're gonna make the app better. You know? Something pretty vague. I said, you know, send me send me your complaints. [00:39:38] S0: I'll write. You're you're pretty... You know, I see you. You're you're sourcing feedback in real time
00:39:38 · Akash Pasricha, Andrew Ambrosino
[00:39:38] S0: I'll write. You're you're pretty... You know, I see you. You're you're sourcing feedback in real time [00:39:42] S5: on social media. And and, you know, like, I... Then I go over to the the app, and I, you know, say to my dot, which runs Codex in the background, obviously. I run... I say to my Dot, hey. Can you take a look at this feedback, thread and save them all, group them into themes, you know, sort them by impact, right, if a if a bunch of people are saying the same thing, but saying them come... Kinda differently. Right? And it kinda goes through. And at the end of the day, I've got this great doc in in space, that has 300 bullet points. And they're all grouped and sorted, we can start to go off that. And so even something like, is that how people thought coding agents would go? I'm not sure. But, you know, it's like every single thing I do now is through this type of experience where I'm like, you know, help me understand this problem that we have. It's not a code problem per se. Some some of it is, but it's it's really it's really interesting. [00:40:35] S0: So when you see headlines like, you know, Bloomberg reported that Apple is is putting caps on how much their employees can spend Yeah. On AI tools, for example. I mean, this is a flavor of headline that we see, you know, every week, and this is really coming down to the cost Right. Of these tools. I mean, what do you think when you see those headlines? [00:41:00] S5: I think a few things. I... The first thing I always think is that I'm surprised that people are surprised. And that there