Latent Space: The AI Engineer Podcast
00:20:18 · Shawn Wang, Mihail Eric
[00:20:18] S0: You know, but it is everything... I'll endorse everything as things that people talk about and you should know. You just... We know the open problems, we may not know the answers. [00:20:27] S1: That's absolutely right. Yeah. Yeah. I mean, even... I mean, you probably have your own version of this. I'd be curious to hear what you think is still defensible, or not defensible, but not as set in stone. But, you know, even like the the softer factory. Are we talking about, like, dark factories? Are we talking about, like, these sort of half lights on factories? Like, I think even there's a spectrum there. Yeah. And there, you know, there's the people that are on the on the very frontier, like like the warps and the factories that are that are, like, building factories and selling them the services. And, you know, even they will only claim, like, 30 some percent, if even that, of of, like, fully, you know, identically driven contributions. So Yeah. What does that mean in the limit? It's... It is hard to say. [00:21:01] S0: And there's people who went very, very far at Dog Factory and then came out saying, was wrong, guys. So we covered that on AIE. Just be aware that not everyone... They probably believe the things they're saying at the time, but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, yeah, I am often wrong. Yeah. Like... And, like, everyone is, like... Because he created a clock code, it's, treating his word as gospel. Totally. And then, no. Well, sometimes he disagrees himself.
00:21:01 · Shawn Wang, Mihail Eric
[00:21:01] S0: And there's people who went very, very far at Dog Factory and then came out saying, was wrong, guys. So we covered that on AIE. Just be aware that not everyone... They probably believe the things they're saying at the time, but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, yeah, I am often wrong. Yeah. Like... And, like, everyone is, like... Because he created a clock code, it's, treating his word as gospel. Totally. And then, no. Well, sometimes he disagrees himself. [00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap.
00:21:29 · Mihail Eric, Shawn Wang
[00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap. [00:22:09] S0: Yeah. I will call out... So some of your... For example, you have Cognition on the week that you're talking about code review. Yeah. I actually did work with Silas on dev and review. [00:22:18] S1: Yeah. Yeah. [00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use
00:22:19 · Shawn Wang, Mihail Eric
[00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use [00:22:29] S1: you guys. That's also something like, yeah. So Monaco does use Devin. And that's... Mean, and Silas actually spoke last year, and so Silas is great. It's just super, super smart. But yeah, we use you guys, and so I was like, okay, among the providers of kind of code review tools that we... And we sampled a lot of them. We actually did kind of converge on Devin for a lot of use cases. [00:22:47] S0: Silas actually more on the model side these days, and then actually it's Walden that you want. [00:22:51] S1: Yeah. Yeah. Yeah. I've spoken to Walden as well. Yeah. [00:22:53] S0: Yeah. I will also shout out Eno from Factory because, you know, talking about agent readiness, they've been all about that. They build signals, tooling on that. Do you guys use that? [00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah.
00:23:02 · Mihail Eric, Shawn Wang
[00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah. [00:23:27] S0: It was fine. I I was at a call. We were talking about this stuff, and, like, we were like, okay. We need to form some opinions about it. And then, like, factory, like, shipped a full product on it that week they were talking about it. I was like, Okay, you won. I mean, everyone kind of agrees on the same things. [00:23:41] S1: Sure, sure. [00:23:41] S0: And it's just a question of, Do you want to turn it into a dashboard? Yeah, yeah, yeah, yeah. Because it's dashboard. [00:23:46] S1: Right, right, right, right, right, right. But you know, value. And I think they did well from a marketing perspective, jumping on the ratings report. And I've definitely spoken to... I will regularly pitch that to other people and say, yeah, you should run ratings reports of your own kind. Of course, now people have open source versions of it, I think the principles are super important.
00:23:46 · Mihail Eric, Shawn Wang
[00:23:46] S1: Right, right, right, right, right, right. But you know, value. And I think they did well from a marketing perspective, jumping on the ratings report. And I've definitely spoken to... I will regularly pitch that to other people and say, yeah, you should run ratings reports of your own kind. Of course, now people have open source versions of it, I think the principles are super important. [00:24:04] S0: Yeah. Do you want to go piece by piece, or do you have any other things? I can go piece by piece. Yeah. Okay. So the first one, right? You focus on Claude Code, but then you also have Pi as an open source sponsor. [00:24:14] S1: Yeah. What's up? Okay. So so so... Okay. This one is maybe a little bit more marketing y, but but... Okay. The one thing I do wanna highlight, and I've made that... I also made that clear last year, but I'm not a... I'm not endorsing any tool. Okay? Mhmm. Like like, I I don't... I I I think that is, the wrong way to approach something like this. There are tools that come and go. When... You know, transparently, when when the class was taught last year, Claude Code was not as popular. It wasn't as, like, really accepted. Boris was only at 4,000 Twitter followers, and now he's at, you know, whatever million. I don't even know if he is, but, you know, like, he's at a lot of followers. So so Claude Code has really, like, emerged on the scene. I think [00:24:50] S0: Yeah. We I think we we also underestimated Claude Code when we had Boris and Kat on the podcast. [00:24:54] S1: Right. Right.