The Information's TITV
00:33:28 · Akash Pasricha, Andrew Ambrosino
[00:33:28] S0: So when people think of Codestorage, I mean, is this like a a a competitor to GitHub? Are you still using GitHub? Just help us understand that a little bit. [00:33:38] S5: Yeah. GitHub is biggest name in Codestorage. We've been a customer for a long time. We still are a customer. We have our open source repos on GitHub. We interact with the community there, Codex being one of those. GitHub is a great partner. Right. I think what we see internally is just, you know, needs that we wanna kinda play with, experiment with, and and see what we can do here. [00:34:02] S0: Right. So the peer acquisition, again, the the conversation around technical challenges, there's so much that's difficult here in what you're building. I wonder if you can talk about some of the other technical challenges that you're seeing, not because things are going wrong, just things that are really difficult to solve right now. I mean, the conversation that I've had with Dan Shipper many times on my show is what what... What's difficult for the models to do right now. And so I I I... In your capacity, it's kind of interesting. You're building this program that the front end is changing. The back end is changing. Tell me about what... What's hard about it right now.
00:34:41 · Andrew Ambrosino
[00:34:41] S5: Yeah. Yeah. The job has changed a lot. Right? And and I think as we've looked at the... You know, we started the... About a year ago, we started building the Codex app, the original version of this, and the models were very, very different at the time. And so we've kinda seen some of these cycles of how software gets developed and how the models are are, you know, bad at something until they're not. And I think, you know, if you if you look at things like plan mode and goal mode and these kind of, you know, featurettes that we've launched, for software developers, some of them are really meant to be temporary. Some of them are meant to say, hey. You know, in April, the model is good at a lot of this stuff, but it it doesn't keep itself going for a long time. Right? So we introduced goal mode, and goal mode is kind of a more deterministic, more structured feature that says, hey. You know, we're gonna we're gonna prompt the model to keep running until it hits this goal. Right? With newer models, you know, with Astra Right. Goal mode's almost not needed anymore. Right? Because it can keep going. But it's kind of always these arcs of of of things where we're like, hey. We're gonna introduce this, like, little helper here in the software. And at some point, we'll be able to retire it because the model will be able to do that kind of natively. Right? I think where we where we still see the the model's struggle overall across the industry is just, like, what does it mean to maintain a product, in a large product and to take in feedback from everybody and to make decisions about what features to prioritize and and what not to and how to design them. And even though the models are writing most of our code, still, you know, this this this aspect of things is is kind of the challenging part.
00:36:19 · Akash Pasricha, Andrew Ambrosino
[00:36:19] S0: This So is kind of interesting. So so in in your capacity, part of developing the product is sometimes maybe compensating for an area where the model might be trying to make a leap, but it... Maybe it's, you know, it's it's not quite there yet, and so you have to sort of work your way around and introduce something temporary to sort of compensate for that. That's that's kind of an interesting way of doing product development. [00:36:44] S5: It is. It is. And we... The most important thing that we do is get out of the way of the models. Right. And make sure that we are developing software that puts everything that the model can do in the hands of as many people as possible in a way that's as clear as possible. Sometimes this means adding affordances for how to use the model. Right? Adding suggestions, adding proactivity, things like this. Right? Sometimes it means, you know, stripping things out when the model isn't needed anymore. There's this kind of classic thing that, like, model over model. The skills need to change. Right? Because you, you know, you make skills for one and it doesn't work for the other. But, yeah, it's a huge part of the job.
00:36:44 · Andrew Ambrosino, Akash Pasricha
[00:36:44] S5: It is. It is. And we... The most important thing that we do is get out of the way of the models. Right. And make sure that we are developing software that puts everything that the model can do in the hands of as many people as possible in a way that's as clear as possible. Sometimes this means adding affordances for how to use the model. Right? Adding suggestions, adding proactivity, things like this. Right? Sometimes it means, you know, stripping things out when the model isn't needed anymore. There's this kind of classic thing that, like, model over model. The skills need to change. Right? Because you, you know, you make skills for one and it doesn't work for the other. But, yeah, it's a huge part of the job. [00:37:20] S0: So in this world, Andrew, where, I mean, so many of these frontier models are are getting so good, and the competition competition for a platform like Codex is is growing by the week, by the month. How do you think about differentiating yourself and and staying differentiated in this world where, you know, people talk about the models being commoditized? Fine. We can have that discussion, really. I mean, this is a... The chat interface, all these different, platforms use. We... You know, how do you think about the nuances that you want to be ahead in the trade offs on the product level?
00:37:56 · Andrew Ambrosino, Akash Pasricha
[00:37:56] S5: Yeah. Yeah. It's a great question. And it's one we think about a lot. We we are very focused on the frontier, and that increasingly means efficiency. Right? So the most powerful models we can possibly make, most efficient token wise and cost wise is where we wanna be. And then on the product side, you know, model to model, I think one of the great strengths that we have internally is that we understand how the model's trained. We can set up flywheels. Right? We can say, hey, the model's gonna be really good at this on its next release, so let's build this feature to really highlight that, you know, which we've done, I think, a few times. But it's it's it's con... It's constant moving. It's it's never something where we say, hey, mission accomplished. We've we've solved this shape of product. That will never be the case with this. Right? Yes, it's a chat interface, but, you know, with with, GPT six, with Intelligent UI, for example, right, now the chat comes back with little widgets. When you ask for, hey. Where's my flight? Right? It might come back with a widget or something like that. So it's it's constantly changing. [00:39:02] S0: And when you are studying customers using Codex and and other products... I mean, let... Let's just stick with Codex for for a moment. There's so much included in there. How is the way that people are using Codex? How has that changed today compared to even six months ago from what you're observing?
00:39:02 · Akash Pasricha, Andrew Ambrosino
[00:39:02] S0: And when you are studying customers using Codex and and other products... I mean, let... Let's just stick with Codex for for a moment. There's so much included in there. How is the way that people are using Codex? How has that changed today compared to even six months ago from what you're observing? [00:39:23] S5: I mean, people use it for absolutely everything now. I mean, it's... Like, I use it to source feedback even. You know? I... The other day, I tweeted something along the lines of, hey. We're gonna make the app better. You know? Something pretty vague. I said, you know, send me send me your complaints. [00:39:38] S0: I'll write. You're you're pretty... You know, I see you. You're you're sourcing feedback in real time