Latent Space: The AI Engineer Podcast
01:43:52 · Shawn Wang, Mihail Eric, S2
[01:43:52] S0: Decide everything for me, please. [01:43:53] S1: Yeah. It is a lot. And and I'm I'm trying to... You know, I haven't, like, gone too deep in understanding, like, the... How they approach their GTM here, but I suspect you would have to do it piecemeal. Right? You would have to be like, hey, we first are your coding agent. Oh, then also... Like, you basically land and expand. Right? I mean, we talked about this bundling and unbundling piece. You don't start by saying, here's your whole factory. There's no way you could get, like, a Ford on board with that. You know what I mean? Where you're gonna go on and just plop the whole thing on their desk. That's at least how I would think about it, is you first be their coding agent of choice. You build up some background agent for them, you own some of their SRE pieces, and then you're like, well, now we can just weave it all together if you define everything in code or something like that. Like, you know, configs. [01:44:32] S2: There's a fun blog post on this from OpenAI. Had My buddy Ryan. We had Ryan on the podcast. Basically they spent five months making an internal tool with zero lines of manually written code. They kind of walk through one, it's called harness engineering. So you can learn harness engineering, but the learnings of there was kind of a slump or slower at first, and then you offload and you had peer review. It's at this point it's from February, but it's a vision of a software factory, right? Like you are full sending, letting agents fully write everything and how to do that well.
01:44:32 · S2, Mihail Eric, Shawn Wang
[01:44:32] S2: There's a fun blog post on this from OpenAI. Had My buddy Ryan. We had Ryan on the podcast. Basically they spent five months making an internal tool with zero lines of manually written code. They kind of walk through one, it's called harness engineering. So you can learn harness engineering, but the learnings of there was kind of a slump or slower at first, and then you offload and you had peer review. It's at this point it's from February, but it's a vision of a software factory, right? Like you are full sending, letting agents fully write everything and how to do that well. [01:45:10] S1: Yeah, absolutely. Yeah. I mean, some version of this is gonna exist. To Swig's point, I do wonder about the the sales piece here, like the actual business of this. That's an interesting question. From a principles perspective, I think this is the the... This is one version of the future. And then beyond that, I think there's gonna be some amount of dabbling into and speaking about, like, okay, what does it mean to... Yeah, what does it mean to be a developer in in the future? Right? Like, are we... People talk about taste. People talk about, like, you know, understanding of systems and architecture and and, yeah, like, software taste. I wanna, like, go a little bit deeper here on maybe starting to, like, at least put some answers to the question around this. [01:45:47] S0: Yeah. So this is your answer to where where AI software engineering goes next.
01:45:47 · Shawn Wang, Mihail Eric
[01:45:47] S0: Yeah. So this is your answer to where where AI software engineering goes next. [01:45:51] S1: Yeah. So, I mean, the factory is is, like, the tactical, but I think beyond that, there's, like, a... There's a question around, like, the role. Right? Like, what does the role evolve into? If if the factory is is running and, you know, full steam ahead doing its thing, the conveyor belt is is churning out code, what what do we do in all this? Right? And I think we... I think you and I, we had this discussion with Jerry at the dinner once upon a time, a few weeks back with Dex and Jerry Yeah. [01:46:12] S0: Yeah. [01:46:12] S1: Yeah. Around the around the factory. But, yeah, I I I think that there was this part of... At least the discussion points there, which I am pretty much on on board with is, like, increasing levels of abstraction, and then somehow there's someone still... Even a CEO of a company has to have enough oversight understanding of, like, different constituent parts to be able to make a judgment on direction. Right? To be able to course correct, to be able to set metrics, and things of that nature. I think there's a version that translates of that. There's a there's a... That analogy that translates to, like, someone who oversees the technical. We call it a manager of agents. We call it a, you know, different terms. They're all buzz buzzword y. But, in some some respects, someone who understands product requirements, constraints, technical constraints, kind of loops through all of those and knows how to adjudicate.
01:46:12 · Mihail Eric, Shawn Wang
[01:46:12] S1: Yeah. Around the around the factory. But, yeah, I I I think that there was this part of... At least the discussion points there, which I am pretty much on on board with is, like, increasing levels of abstraction, and then somehow there's someone still... Even a CEO of a company has to have enough oversight understanding of, like, different constituent parts to be able to make a judgment on direction. Right? To be able to course correct, to be able to set metrics, and things of that nature. I think there's a version that translates of that. There's a there's a... That analogy that translates to, like, someone who oversees the technical. We call it a manager of agents. We call it a, you know, different terms. They're all buzz buzzword y. But, in some some respects, someone who understands product requirements, constraints, technical constraints, kind of loops through all of those and knows how to adjudicate. [01:46:53] S0: Yeah. I will say that in the chair before you, we just had Anthropics head of products and head of engineering for the And engineering I was like, When your jobs gonna come by? And they're like, Not yet. And so if Anthropic's not there yet, you should not feel like it's imminent. Good, good, good. Did they say how long or are they just not yet? Not yet, but who knows? Definitely... There was definitely a lot of like, okay. Well, yeah, I'm doing her job. She's doing my job. Like, yeah. Yeah. I think it's
01:46:53 · Shawn Wang, S2
[01:46:53] S0: Yeah. I will say that in the chair before you, we just had Anthropics head of products and head of engineering for the And engineering I was like, When your jobs gonna come by? And they're like, Not yet. And so if Anthropic's not there yet, you should not feel like it's imminent. Good, good, good. Did they say how long or are they just not yet? Not yet, but who knows? Definitely... There was definitely a lot of like, okay. Well, yeah, I'm doing her job. She's doing my job. Like, yeah. Yeah. I think it's [01:47:27] S2: also interesting perspective for students. Right? You said there's a lot of sophomore undergrads. Like, you got two, three years to go to plan out what you wanna be doing. There's different ways to look at it from that, right? You're not straight in industry.
01:47:27 · S2, Shawn Wang
[01:47:27] S2: also interesting perspective for students. Right? You said there's a lot of sophomore undergrads. Like, you got two, three years to go to plan out what you wanna be doing. There's different ways to look at it from that, right? You're not straight in industry. [01:47:39] S0: The thing that I care... I'm thinking a lot about is, well, so one part of this discussion that resonates more with me is the loop stuff Mhmm. Where, like, okay, you you started writing prompts. Now you no longer write prompts. You write prompts. They write prompts. That's right. Right? And then you then you just keep going up. And I I think to me, the sort of recursive concentric circles nature going outward, but also going inward is something that builds the factory. I think you build up to the factory, not build down. That's right. But I think a lot of people will eventually start building down. Like you start from the top level, like what is my goal? And then you figure out the systems. That currently creates a ton of slop and you should not do that. But someday you will. Yeah. So instead of, so that will be my question to prompt you is do we... What is the sort of prime question that people exist to solve? Like, that is your thing, right? So let's say an engineer today, you are measured by your PR merge rate, PR contribution rate. Well, two years from now, that's gonna be ridiculous. Okay. Well, what's your main thing? I don't know. Money? People always go to money too quickly. It's something in between money and