Latent Space: The AI Engineer Podcast
00:28:56 · Mihail Eric, Shawn Wang, S2
[00:28:56] S1: Yeah, I think it's a bit self selecting because the curriculum is all here. So the curriculum has been available since, mean, left at least a few weeks. And I guess I announced it a few weeks ago, but it had been in the works for probably least one to two months. So if you looked for it, you could probably find it. And I feel like the reality is if students are not interested in this, then they won't... They just won't show up. They choose not to show up. What I'm finding that is interesting is there are definitely students that show up, but then there's also just a lot of from, you know, broader Stanford postdocs at other departments. There's a lot of, like, interdisciplinary sort of interests where folks that are in, you know, somewhere in pathology or, like, you know, medicinal sort of fields or or even, like, random other engineering fields that aren't pure CS that are coming in and also interested in kind of getting exposed to this. Also a lot of, GSB students, MBAs, that also, wanna, I guess, learn how to not have a technical co founder or whatever the motivation there is. [00:29:52] S0: Yeah. So I I think... By the way, if anyone hasn't done the MITM proxy exercise Yeah. You should do it. [00:29:58] S1: Yes. [00:29:58] S0: You'll be shocked. Claude Code has the longest system prompt of all the coding agents, something like 20,000 tokens. [00:30:05] S1: Yeah. Yeah. Although, yeah. Yeah. That is true. That is true. Yeah. Yeah. Although they've also reduced [00:30:09] S2: the size. [00:30:09] S0: They cut it [00:30:09] S2: by 80%.
00:30:09 · S2, Shawn Wang, Mihail Eric
[00:30:09] S2: by 80%. [00:30:10] S0: Yeah. Yeah. I know. But then they grow again, then they cut and then grow again. [00:30:14] S1: It's it's a seesaw. [00:30:15] S0: Yeah. Yeah. No. I I think it's interesting. You'll be surprised, like, what goes in there that, actually, sometimes you don't use. Mhmm. Sometimes it actually hints at the internal workings of some of the tools. Anything that you found there that could be cool to talk about?
00:30:28 · Mihail Eric
[00:30:28] S1: Yeah. No. I mean, in fact, yesterday when I was talking about this, it was a little bit of this, like, one of the things that I thought was most interesting about Claude Code back in the day, and I think part of why it really blew up in terms of popularity, was their focus on simplicity. Right? So in the early days, they're among the first to really lean into this, let's fewer tools do more. Right? Lean into SQL, lean into Bash, lean into, like, the things that are that are powerful, like Swiss Army knife tools, and have that be a lot of the driver of capability. I think they did a fantastic job leaning to that. I think that's why they they were as popular as they did. I think that when I presented something like the system prompt, it was like, well, there was a different system prompt about two months ago. It is now... They have reduced in complexity, and it turns out that now what was once part of the system prompt, things like how to use even Git, right, or how to, like, set up tasks things of that nature are no longer part of the system prompt. It's actually expected to be more kind of captured in the models, like the model weights itself. So they've kind of baked in a lot of those learnings more at the model level. And that just raises interesting questions around, you know, the model versus the harness. Right? And how do you kind of think through that trade off? What do companies invest in? What should you invest in to really push the frontier of your own capabilities? And so I I think we still, even as a as a community, we seesaw between the two a lot.
00:31:39 · S2, Mihail Eric, Shawn Wang
[00:31:39] S2: I think you don't want to fault them for growing a system prompt and then cutting it. At some level, it's also how the model was, right? Weaker older models, you need to give them examples of this is good, this is bad. And then obviously you cut that away at some point, right? And that's just part of learning what has been done, right? You can apply the same towards, if you're using a local coding agent with not the strongest model, you might need to add these examples, right? [00:32:05] S1: That's right. Is that correct? Yeah. Yeah. And then, of course, there's interesting AI learnings to fall out of that, which is why, as the models get better, when are you over constraining it by adding those examples, right? [00:32:16] S0: And then how you [00:32:16] S1: can pull away the over constraining. That's, I think, part of the learning as well. [00:32:20] S0: Okay. Week two, you're going to talk about context engineering. Repet, I haven't seen, but there's many versions of spectrum development. What would you highlight to a broader audience about what you're putting in here that people should know? [00:32:35] S2: Yeah. This is like the one hour version of the course. Hey, exactly. [00:32:40] S0: Please condense everything. The answer [00:32:42] S1: is going just... That's fair, yeah. Like... [00:32:46] S0: Well, yeah. It's it's also a trailer. Right? It's like, well, if you want more... [00:32:48] S2: If you're signing up next next year or next fall for courses, you know, this is your one hour.
00:32:48 · S2, Mihail Eric, Shawn Wang
[00:32:48] S2: If you're signing up next next year or next fall for courses, you know, this is your one hour. [00:32:53] S1: This is like this is like... Almost like the quibi of the class. You know I mean? Where whether it's it's too long to be a trailer. It's too short to be the class. Is this like... What I guess people will double best part, [00:33:02] S2: if it's a trailer, by the next time you teach it, it'll change again. [00:33:05] S1: That's true, [00:33:06] S0: that's true,
00:33:07 · Mihail Eric
[00:33:07] S1: that's true. Yeah. Well, so it's okay. So a good call out on engineers, it's funny. I think that the main thing that I just wanna reinforce here is, and I see this even when I'm, like, interviewing folks. Very few people in industry even progress beyond, like, just vanilla prompting. Right? There's, like, a form of, like, hey, do me the thing. Right? Like, do do the thing. And and that, certainly as the models get good, that can carry you a certain distance. But inevitably, you know, any folks that I speak to that are... Come out of, like, a Google or some place where there's a massive, like, code base, the reality is you have to be lot more thoughtful about how you're, like, ingesting context. Right? And so thinking through what are some of those principles and those techniques around how to, like, point, you know... And I sort of, you know, the rapid framework that I'm sort of describing here is is functionally similar to, like, DEX's RPI, right, or something of that nature, where you have this, like, re research kind of proposal, and sort of, like, a spec, and then and then you do, like, your implementation. And then you kind of tie together with some testing and whatnot. So what I just wanted to reinforce here is that, like, you should have this step, especially when you're entering unfamiliar code bases, when you're trying to, sort of infuse into the context something that an agent needs to understand about a very large code base. There's value in something that is a research phase. And then in the case of this particular sort of framework that I'm putting forward, it's also then suggests some proposal where there's a few alternative ways of solving a problem once you've done the research. And so this is where I also invite the human in the loop to decision be maker on one approach versus the other. And then you enter the plan, which is kind of the spec. All to say, the TLDR is like, there is a way to do this that actually does scale out to code bases that are millions of lines long. And it's not just the vanilla prompt. And I think that's what I wanna highlight. If you're gonna go work in these big code bases, you should use something a little bit more involved.