Latent Space: The AI Engineer Podcast
01:42:22 · S2, Mihail Eric, Shawn Wang
[01:42:22] S2: I mean, it's fun a time. What's the future? I feel like all the guys can adjust all the things. [01:42:27] S1: Yeah, yeah, yeah. So there's a few aspects here. Okay, so there's little bit of spillover from the previous week. So yeah, coding agents, big teams, some of the concepts, but then, you know, we'll touch on the software factory. I want people to really start thinking about this world of this machine that builds the machine, and where, if at all, are people still in the loop there. Today, I think the answer is some combination of intent kind of review, or maybe the places, like, intent where you're defining the specs and what needs to be built, and review where you're doing some amount of your own checking of the system. And of course, also... This is, I think, that bottleneck is gonna be loosened up before even the intent piece does, but for now, are the two toe points, I think, in a fully automated system. So yeah, I just wanna get people exposed to that. For one of the USS partners, actually, WARP, they're doing this thing as part of their contribution to the USS program, where they're gonna essentially make a version of WARP that is running for the factory, and students will just be building the factory, contributing to the factory, and keep [01:43:23] S0: it running. Like, work in [01:43:24] S1: our factory. Work in our factory. Exactly. And so that's gonna be one of the ways... And then they're gonna, yeah, find things that they find that are interesting there and pull it back in.
01:43:24 · Mihail Eric, Shawn Wang
[01:43:24] S1: our factory. Work in our factory. Exactly. And so that's gonna be one of the ways... And then they're gonna, yeah, find things that they find that are interesting there and pull it back in. [01:43:30] S0: This is one of those things where, like, I'm not sure, Obviously, factory itself is very, very keen on this. [01:43:35] S1: Yeah. I [01:43:36] S0: feel like everyone wants to build their own. [01:43:38] S1: Yeah. I I think that's... That is... It it is a good question. And I think they're, you of course, entitled to kind of try and sell the products they do. I think it's a hard system to own its entirety. It's a it's a lot. It's a lot. [01:43:52] S0: Decide everything for me, please.
01:43:52 · Shawn Wang, Mihail Eric, S2
[01:43:52] S0: Decide everything for me, please. [01:43:53] S1: Yeah. It is a lot. And and I'm I'm trying to... You know, I haven't, like, gone too deep in understanding, like, the... How they approach their GTM here, but I suspect you would have to do it piecemeal. Right? You would have to be like, hey, we first are your coding agent. Oh, then also... Like, you basically land and expand. Right? I mean, we talked about this bundling and unbundling piece. You don't start by saying, here's your whole factory. There's no way you could get, like, a Ford on board with that. You know what I mean? Where you're gonna go on and just plop the whole thing on their desk. That's at least how I would think about it, is you first be their coding agent of choice. You build up some background agent for them, you own some of their SRE pieces, and then you're like, well, now we can just weave it all together if you define everything in code or something like that. Like, you know, configs. [01:44:32] S2: There's a fun blog post on this from OpenAI. Had My buddy Ryan. We had Ryan on the podcast. Basically they spent five months making an internal tool with zero lines of manually written code. They kind of walk through one, it's called harness engineering. So you can learn harness engineering, but the learnings of there was kind of a slump or slower at first, and then you offload and you had peer review. It's at this point it's from February, but it's a vision of a software factory, right? Like you are full sending, letting agents fully write everything and how to do that well.
01:44:32 · S2, Mihail Eric, Shawn Wang
[01:44:32] S2: There's a fun blog post on this from OpenAI. Had My buddy Ryan. We had Ryan on the podcast. Basically they spent five months making an internal tool with zero lines of manually written code. They kind of walk through one, it's called harness engineering. So you can learn harness engineering, but the learnings of there was kind of a slump or slower at first, and then you offload and you had peer review. It's at this point it's from February, but it's a vision of a software factory, right? Like you are full sending, letting agents fully write everything and how to do that well. [01:45:10] S1: Yeah, absolutely. Yeah. I mean, some version of this is gonna exist. To Swig's point, I do wonder about the the sales piece here, like the actual business of this. That's an interesting question. From a principles perspective, I think this is the the... This is one version of the future. And then beyond that, I think there's gonna be some amount of dabbling into and speaking about, like, okay, what does it mean to... Yeah, what does it mean to be a developer in in the future? Right? Like, are we... People talk about taste. People talk about, like, you know, understanding of systems and architecture and and, yeah, like, software taste. I wanna, like, go a little bit deeper here on maybe starting to, like, at least put some answers to the question around this. [01:45:47] S0: Yeah. So this is your answer to where where AI software engineering goes next.
01:45:47 · Shawn Wang, Mihail Eric
[01:45:47] S0: Yeah. So this is your answer to where where AI software engineering goes next. [01:45:51] S1: Yeah. So, I mean, the factory is is, like, the tactical, but I think beyond that, there's, like, a... There's a question around, like, the role. Right? Like, what does the role evolve into? If if the factory is is running and, you know, full steam ahead doing its thing, the conveyor belt is is churning out code, what what do we do in all this? Right? And I think we... I think you and I, we had this discussion with Jerry at the dinner once upon a time, a few weeks back with Dex and Jerry Yeah. [01:46:12] S0: Yeah. [01:46:12] S1: Yeah. Around the around the factory. But, yeah, I I I think that there was this part of... At least the discussion points there, which I am pretty much on on board with is, like, increasing levels of abstraction, and then somehow there's someone still... Even a CEO of a company has to have enough oversight understanding of, like, different constituent parts to be able to make a judgment on direction. Right? To be able to course correct, to be able to set metrics, and things of that nature. I think there's a version that translates of that. There's a there's a... That analogy that translates to, like, someone who oversees the technical. We call it a manager of agents. We call it a, you know, different terms. They're all buzz buzzword y. But, in some some respects, someone who understands product requirements, constraints, technical constraints, kind of loops through all of those and knows how to adjudicate.
01:46:12 · Mihail Eric, Shawn Wang
[01:46:12] S1: Yeah. Around the around the factory. But, yeah, I I I think that there was this part of... At least the discussion points there, which I am pretty much on on board with is, like, increasing levels of abstraction, and then somehow there's someone still... Even a CEO of a company has to have enough oversight understanding of, like, different constituent parts to be able to make a judgment on direction. Right? To be able to course correct, to be able to set metrics, and things of that nature. I think there's a version that translates of that. There's a there's a... That analogy that translates to, like, someone who oversees the technical. We call it a manager of agents. We call it a, you know, different terms. They're all buzz buzzword y. But, in some some respects, someone who understands product requirements, constraints, technical constraints, kind of loops through all of those and knows how to adjudicate. [01:46:53] S0: Yeah. I will say that in the chair before you, we just had Anthropics head of products and head of engineering for the And engineering I was like, When your jobs gonna come by? And they're like, Not yet. And so if Anthropic's not there yet, you should not feel like it's imminent. Good, good, good. Did they say how long or are they just not yet? Not yet, but who knows? Definitely... There was definitely a lot of like, okay. Well, yeah, I'm doing her job. She's doing my job. Like, yeah. Yeah. I think it's