Latent Space: The AI Engineer Podcast
00:21:29 · Mihail Eric, Shawn Wang
[00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap. [00:22:09] S0: Yeah. I will call out... So some of your... For example, you have Cognition on the week that you're talking about code review. Yeah. I actually did work with Silas on dev and review. [00:22:18] S1: Yeah. Yeah. [00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use
00:22:19 · Shawn Wang, Mihail Eric
[00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use [00:22:29] S1: you guys. That's also something like, yeah. So Monaco does use Devin. And that's... Mean, and Silas actually spoke last year, and so Silas is great. It's just super, super smart. But yeah, we use you guys, and so I was like, okay, among the providers of kind of code review tools that we... And we sampled a lot of them. We actually did kind of converge on Devin for a lot of use cases. [00:22:47] S0: Silas actually more on the model side these days, and then actually it's Walden that you want. [00:22:51] S1: Yeah. Yeah. Yeah. I've spoken to Walden as well. Yeah. [00:22:53] S0: Yeah. I will also shout out Eno from Factory because, you know, talking about agent readiness, they've been all about that. They build signals, tooling on that. Do you guys use that? [00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah.
00:23:02 · Mihail Eric, Shawn Wang
[00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah. [00:23:27] S0: It was fine. I I was at a call. We were talking about this stuff, and, like, we were like, okay. We need to form some opinions about it. And then, like, factory, like, shipped a full product on it that week they were talking about it. I was like, Okay, you won. I mean, everyone kind of agrees on the same things. [00:23:41] S1: Sure, sure. [00:23:41] S0: And it's just a question of, Do you want to turn it into a dashboard? Yeah, yeah, yeah, yeah. Because it's dashboard. [00:23:46] S1: Right, right, right, right, right, right. But you know, value. And I think they did well from a marketing perspective, jumping on the ratings report. And I've definitely spoken to... I will regularly pitch that to other people and say, yeah, you should run ratings reports of your own kind. Of course, now people have open source versions of it, I think the principles are super important.
00:23:46 · Mihail Eric, Shawn Wang
[00:23:46] S1: Right, right, right, right, right, right. But you know, value. And I think they did well from a marketing perspective, jumping on the ratings report. And I've definitely spoken to... I will regularly pitch that to other people and say, yeah, you should run ratings reports of your own kind. Of course, now people have open source versions of it, I think the principles are super important. [00:24:04] S0: Yeah. Do you want to go piece by piece, or do you have any other things? I can go piece by piece. Yeah. Okay. So the first one, right? You focus on Claude Code, but then you also have Pi as an open source sponsor. [00:24:14] S1: Yeah. What's up? Okay. So so so... Okay. This one is maybe a little bit more marketing y, but but... Okay. The one thing I do wanna highlight, and I've made that... I also made that clear last year, but I'm not a... I'm not endorsing any tool. Okay? Mhmm. Like like, I I don't... I I I think that is, the wrong way to approach something like this. There are tools that come and go. When... You know, transparently, when when the class was taught last year, Claude Code was not as popular. It wasn't as, like, really accepted. Boris was only at 4,000 Twitter followers, and now he's at, you know, whatever million. I don't even know if he is, but, you know, like, he's at a lot of followers. So so Claude Code has really, like, emerged on the scene. I think [00:24:50] S0: Yeah. We I think we we also underestimated Claude Code when we had Boris and Kat on the podcast. [00:24:54] S1: Right. Right.
00:24:54 · Mihail Eric, Shawn Wang, S2
[00:24:54] S1: Right. Right. [00:24:54] S0: It was, like, two months old. I hadn't really... Like, I used it once. A query cost me 50¢. I was like, who's gonna use this shit? Totally. [00:25:00] S2: Totally. It was still in beta for a while, but I think it's good on them for how they built it before the model was ready. Yes. Like you hear so much builds for where the model will go. [00:25:08] S1: Totally. [00:25:09] S2: Super ambitious. Models were not that good, you couldn't feel it, and then, ah, we see that's where
00:25:14 · Mihail Eric
[00:25:14] S1: it came from. Yeah, it's funny because when he gave his talk last year, I remember, it was like a very high level sort of set of slides, but I remember one of the slides was literally built for where the model's gonna be six months from now. And this was, like, October of last year, and I can see why he would've made that point if if he knew in six months where, like, Opus was gonna be, because that was also what I think was a huge unlock where I would argue was one of the reasons why the curriculum changed as much as it did. Right? Where there was certainly this huge arms race around true, like, autonomy in these agents, like, running agentic coding agent tasks. But... Yeah. So so, like, you know, back to your point, it's, like, less on... I'm not endorsing Cloud... Mean, you know, Claude Code, of course, I use it and and use other things as well. But but here, I was more about, like, the... What does a modern coding harness look like? I wrote a blog post that actually, the... Yeah. This one, where this was, yeah, earlier this year, where I sort of broke down some of... Some similar ish principles. The TLDR was that, you know, you could sort of, through, like, pretty much introductory algorithms level understanding, you could just find, like, essentially an agentic loop that does the bulk of what you would call a coding agent. And, of course, there's, like, bells and whistles. There's no sub agents. There's no MCP, none of that stuff. But if you wanna see something that you talk to, and then it literally will edit your file, and you can do pretty sophisticated things with it, you... It doesn't require more than 200 lines. So I... You know, a lot of what I've tried to do in education is to try to, like, peel away the stuff that's like the secret sauce and show that really at its core, I mean, are still students and you want