Latent Space: The AI Engineer Podcast
00:34:51 · S2, Shawn Wang
[00:34:51] S2: Anything to add on context engineering? [00:34:54] S0: I think that probably the weirdest thing about how I think about context right now is when you split out to a sub agent, [00:35:02] S2: because [00:35:04] S0: subagents can benefit from inheriting your entire context of the main thread or not. Sometimes there's a regular disagreement on on that. And then sometimes you want to condense what the subagent does and researchers put it back onto the main thread. And it's very hacky. I don't like anything I'm doing. So I don't have any answers. All I know is, like, when you want to fan out work Mhmm. It is not just about you have a big spec and you you, you know, you do RPI, you do RREPIT, whatever. It is not one agent anymore. It is about fanning out work and making sure they all have the right context and they all have the right guidance. If you have a review agent reviewing all the other works, you have... Make that guy have have everything. And you always go through this phase of expanding contracts, like, oh my god. These guys don't have enough context. I'm gonna give them more context. Okay. Great. And then they're gonna overture you into some random one off comment that you made a while ago and then make some really stupid mistakes because you you didn't mean it that way, but, like, like, took it as the law. Right. Something like that. And so I don't know what what the word is I'm saying. Right? This is, like, multi agent management for context.
00:36:12 · Mihail Eric, Shawn Wang, S2
[00:36:12] S1: Yeah. Yeah. Yeah. Yeah. I think that that's absolutely right. There... There's, like, some aspect of, this is maybe the part that's more, maybe, among the most art than science. Right? It's like some combination, I think, to your point. Yes, the fan out for the exploration, the fan out for the subtasks, then the consolidation, compaction, whatever you might want to call it. I think being thoughtful about refreshing of the context, understanding when things are getting a little bit too bloated. I think Dex had this concept of the dumb zone and sort of being a little bit more thoughtful about the dumb zone. [00:36:39] S0: He's good at these equating things. [00:36:40] S1: Oh, Yeah. [00:36:40] S2: Think The little diagrams. [00:36:41] S1: The little diagrams. Oh, mean How did you [00:36:43] S0: have him on? [00:36:44] S1: Yeah. Well, think he's... I mean, he definitely informed a lot of the curriculum here. And so he's... Which is fantastic, I think, to think around all this. No, I think the diagrams are huge. That's useful.
00:36:53 · S2, Mihail Eric
[00:36:53] S2: I would add, breadcrumbs for people that are trying to look into this. It's good to see best context engineering preferences, like on your sub agent inheriting cloud managed agents has a version where sessions are separate and it's not inherited. This used to be a big topic before reasoning models when agents didn't run as long. We had shorter context windows. Everyone knows very, very managing that precious context window, right? LingQing used to write a lot of blogs about this in good context engineering. All this stuff is good for people to know. I think in the student context as well, you can expect the average student to take your class because it's a great class and they'll know all these things. You should know them, know them better as well. This is a fun space for research as well. One of the papers, research papers presented at AIE, they looked at the Delta in the Claude code system prompt, what got changed. And they plotted a whole bunch of changes in performance. They tried to reverse engineer it. It's just good research that you can do. There's another one that also looked at context window management and how it affects performance downstream. They weren't the best papers, but like it's quite accessible and it puts out good blog posts. So people that are trying to work on it, I just recommend it. It's a good area you can easily poke into. [00:38:15] S1: Yeah. Take a shout out. [00:38:16] S2: Agent skills, CLI?
00:38:17 · Mihail Eric
[00:38:17] S1: Yeah. Yeah. And I think all this is a... So a little bit of an extension from a single kind of function tool calling, MCP, right, using... Interacting with third party external services, and then extending into what you would call procedures. Like, how do you automate some set of actions over and over again that you then formalize in the form of a SkillMD or something like that? So I think just making that a little bit more, giving people exercise in codifying some of those procedures. I think what I'll also sort of include here is a few kind of fun skills that I think are good to use. Like, I think Matt Pocock's, like, real knee is a good one. And then I... There's ones that I also, from... More from the educational perspective, like Geoffrey Litt. He has this, like, explain diff, which I think becomes especially interesting once you start talking about code review because, you know, the whole Understand the code. Understand the code. Exactly. Yeah. You gotta understand the code. And and, I don't know if you guys have used that, but effectively, like, you run... You know, you have your your your kind of... The the the diff you've put in your code, and then you run the skill. And at the end of it, you have this... Not only, like, kind of a visual representation of what's happened, so you can, you know, kind of skim it and understand what's what's actually at the core of it. But then, what I really liked was this almost, like, quiz at the end of, at the end of it, where you go through and sort of, like, really reinforce what you understand what's happening. And I think that that level of interactive sort of back and forth, I think, is quite nice.
00:39:34 · Shawn Wang, Mihail Eric, S2
[00:39:34] S0: Yeah. There's there's a couple blog posts in this domain that I really like to bring up. There's one, I think, recently, which is, like, kind of... There's no situation in which turning your brain off is a good idea. Like, you know? And and I think these skills actually force you to turn brain on. And when everyone wants to use coding, just turn brain off. They're just like, me an app and get... Read my mind and guess... Get everything correct without me telling you about it. [00:39:57] S1: That's right. [00:39:58] S0: But, like, a lot of these skills are mostly in the genre of like, no, no, like ask me more questions, please. Like Yeah. [00:40:04] S2: I think there's a lot more output than people are used to. Like I can type in a one line prompt and get tens of thousands of lines of code and a whole structure. [00:40:13] S0: It's like, look at my productivity guys. [00:40:16] S2: But more effectively you should know what you're working on. You can really be put out of touch out of the loop. You should spend more time thinking about what you're actually doing. Doing the work and getting the code written is much easier than it was. [00:40:33] S1: Yeah. Yeah. The understanding there's no no excuse for, for sure. [00:40:36] S0: Sorry. What is a web skill? [00:40:38] S1: Yeah. So, I mean, effectively, the skills that interact with, like, the web itself. With computer use. Computer use. Yeah. Things like this. [00:40:44] S0: Anything else? [00:40:45] S1: I mean, mainly things that involve, like, pulling, pulling information from the web. I think it's...
00:40:45 · Mihail Eric, S2, Shawn Wang
[00:40:45] S1: I mean, mainly things that involve, like, pulling, pulling information from the web. I think it's... [00:40:49] S2: Okay. Are there best practices on skills that will interact with the web versus... [00:40:53] S1: I think this is more of, like, a bit of a survey or kind of a demo of stuff that I found that was kind of interesting. I think it was... It might have been out of... Was it like a Vercel? I'm trying to remember now. I'd have to look back at the the full, like, reading materials. But there was, like, one site that I found where I think they... Maybe it was... It might have been browser based, actually. They might have like done... Do you remember this? [00:41:12] S0: Yeah. Well, it would be them that had coined this thing. It's just that I was like, did I miss something? This was like a big part of No, the skills [00:41:19] S2: he's good. He's being educated. Yeah. [00:41:23] S1: I think they made some sort of... Because I guess they were... Yeah. I mean, browsers, obviously, like... They like browsers. They like browsers. And, but, they they had, like, a good, I think, website where they showed some of these interesting capabilities. [00:41:33] S2: Seems legit. CLI is a fun one. Do you have a lot of students, or this is their first time in a terminal in a CLI? Or is it kind of the norm now?