Latent Space: The AI Engineer Podcast
00:24:54 · Mihail Eric, Shawn Wang, S2
[00:24:54] S1: Right. Right. [00:24:54] S0: It was, like, two months old. I hadn't really... Like, I used it once. A query cost me 50¢. I was like, who's gonna use this shit? Totally. [00:25:00] S2: Totally. It was still in beta for a while, but I think it's good on them for how they built it before the model was ready. Yes. Like you hear so much builds for where the model will go. [00:25:08] S1: Totally. [00:25:09] S2: Super ambitious. Models were not that good, you couldn't feel it, and then, ah, we see that's where
00:25:14 · Mihail Eric
[00:25:14] S1: it came from. Yeah, it's funny because when he gave his talk last year, I remember, it was like a very high level sort of set of slides, but I remember one of the slides was literally built for where the model's gonna be six months from now. And this was, like, October of last year, and I can see why he would've made that point if if he knew in six months where, like, Opus was gonna be, because that was also what I think was a huge unlock where I would argue was one of the reasons why the curriculum changed as much as it did. Right? Where there was certainly this huge arms race around true, like, autonomy in these agents, like, running agentic coding agent tasks. But... Yeah. So so, like, you know, back to your point, it's, like, less on... I'm not endorsing Cloud... Mean, you know, Claude Code, of course, I use it and and use other things as well. But but here, I was more about, like, the... What does a modern coding harness look like? I wrote a blog post that actually, the... Yeah. This one, where this was, yeah, earlier this year, where I sort of broke down some of... Some similar ish principles. The TLDR was that, you know, you could sort of, through, like, pretty much introductory algorithms level understanding, you could just find, like, essentially an agentic loop that does the bulk of what you would call a coding agent. And, of course, there's, like, bells and whistles. There's no sub agents. There's no MCP, none of that stuff. But if you wanna see something that you talk to, and then it literally will edit your file, and you can do pretty sophisticated things with it, you... It doesn't require more than 200 lines. So I... You know, a lot of what I've tried to do in education is to try to, like, peel away the stuff that's like the secret sauce and show that really at its core, I mean, are still students and you want
00:26:35 · Shawn Wang, Mihail Eric
[00:26:35] S0: to teach [00:26:35] S1: conceptual principles. There's nothing more to this than... In Stanford, there's this class called one zero six b. It's like the introductory to... In search algorithms. You don't need more than one zero six b to build something like this. And so there you go. There is. [00:26:47] S0: Given that you already taught this class, any pushback, any questions, any weirdness that you observed from the students? On What do they not get? What do they ask seem to cluster questions around? [00:26:58] S1: You mean, like, now that this... Like, this particular concept or just in general? [00:27:02] S0: This concept. Of of, like... Of, like, of, like, building it from scratch and then, like, doing doing what you did. Because this is basically what you needed for Pi. Right. [00:27:10] S1: Right. Right. Exactly. [00:27:11] S0: Right. Yeah. Yeah.
00:27:11 · Shawn Wang, Mihail Eric
[00:27:11] S0: Right. Yeah. Yeah. [00:27:12] S1: Right. So, yeah. It's a... It it... Not really. I think they're... I haven't got... Received any, like, pushback on on, like, this idea at all. I think they're actually more just... Most of them are consumers. Right? Most of most of the students do just, end up using the tools. I I don't think that they get a lot of exposure, if any, certainly in the coursework around peeling that onion back. Right? And I think... So this is the week of the harness. So so so Tuesday, first lecture, you know, logistics, blah blah blah, all that stuff. But then first half is... The first true bit of curriculum is the 200 lines of a harness. And then, yesterday, when I had sort of the follow-up to that, is we actually took Claude Code, and then you have these proxy things, these proxy tools where you can intercept [00:27:53] S0: the request. The MITM proxy?
00:27:53 · Shawn Wang, Mihail Eric, S2
[00:27:53] S0: the request. The MITM proxy? [00:27:54] S1: Yes. Exactly. Exactly right. So use that. Use that tool, intercept the requests and responses. Nice. Look under the hood, see what's happening there. And then then just like, okay, I did say you have Claude code in 200 lines, but really, okay, here's like actual Claude code. And so what I literally did is I took Opus five fives, this model of two days old, and was like, this is... I intercepted those requests. Here's exactly what's going on. Here's the prompt. Here's the system prompt. Here's the plan prompt. Here's the compaction prompt. And then we just went through all of that and sort of showed it to the students. So now they really, really have a state of the art harness that they can understand. [00:28:24] S2: Yeah. It's nice to get those three levels. Students are very much consumers. Here's the baseline agent in a loop and then the real one. I like how you have a video walkthrough of your blog post as well. Do you see students are on like extremes of some people are power users. They consume a lot for some people. You could assume that this is kind of very straightforward if you know how agents work. How do you kind of balance that in the classroom setting? Do you tailor towards the average?
00:28:56 · Mihail Eric, Shawn Wang, S2
[00:28:56] S1: Yeah, I think it's a bit self selecting because the curriculum is all here. So the curriculum has been available since, mean, left at least a few weeks. And I guess I announced it a few weeks ago, but it had been in the works for probably least one to two months. So if you looked for it, you could probably find it. And I feel like the reality is if students are not interested in this, then they won't... They just won't show up. They choose not to show up. What I'm finding that is interesting is there are definitely students that show up, but then there's also just a lot of from, you know, broader Stanford postdocs at other departments. There's a lot of, like, interdisciplinary sort of interests where folks that are in, you know, somewhere in pathology or, like, you know, medicinal sort of fields or or even, like, random other engineering fields that aren't pure CS that are coming in and also interested in kind of getting exposed to this. Also a lot of, GSB students, MBAs, that also, wanna, I guess, learn how to not have a technical co founder or whatever the motivation there is. [00:29:52] S0: Yeah. So I I think... By the way, if anyone hasn't done the MITM proxy exercise Yeah. You should do it. [00:29:58] S1: Yes. [00:29:58] S0: You'll be shocked. Claude Code has the longest system prompt of all the coding agents, something like 20,000 tokens. [00:30:05] S1: Yeah. Yeah. Although, yeah. Yeah. That is true. That is true. Yeah. Yeah. Although they've also reduced [00:30:09] S2: the size. [00:30:09] S0: They cut it [00:30:09] S2: by 80%.