Latent Space: The AI Engineer Podcast
00:19:28 · Mihail Eric, Shawn Wang
What should students think about in terms of defensible skill set that maybe wasn't true before? And so a lot more team dynamic type considerations. And then, yeah, this is a... It all kind of builds up toward the toward the factory, which I think is top of mind for now, for a lot of folks today. Whence we're we're building factories, where do humans still exist and all that? What is the... You know, where are the new choke points that that are intentionally there around around human involvement? Things like intent, things like setting up a verifiable systems, and what happens in the limit when maybe 89% of code is fully driven by just agents, where we kind of filled into all that. So, yeah. [00:20:04] S0: Nice solid 10. I was gonna say, like, you know, some of these are more established patterns than others. And I think people need to know that not all of this is settled science. [00:20:18] S1: True. [00:20:18] S0: You know, but it is everything... I'll endorse everything as things that people talk about and you should know. You just... We know the open problems, we may not know the answers.
00:20:18 · Shawn Wang, Mihail Eric
[00:20:18] S0: You know, but it is everything... I'll endorse everything as things that people talk about and you should know. You just... We know the open problems, we may not know the answers. [00:20:27] S1: That's absolutely right. Yeah. Yeah. I mean, even... I mean, you probably have your own version of this. I'd be curious to hear what you think is still defensible, or not defensible, but not as set in stone. But, you know, even like the the softer factory. Are we talking about, like, dark factories? Are we talking about, like, these sort of half lights on factories? Like, I think even there's a spectrum there. Yeah. And there, you know, there's the people that are on the on the very frontier, like like the warps and the factories that are that are, like, building factories and selling them the services. And, you know, even they will only claim, like, 30 some percent, if even that, of of, like, fully, you know, identically driven contributions. So Yeah. What does that mean in the limit? It's... It is hard to say. [00:21:01] S0: And there's people who went very, very far at Dog Factory and then came out saying, was wrong, guys. So we covered that on AIE. Just be aware that not everyone... They probably believe the things they're saying at the time, but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, yeah, I am often wrong. Yeah. Like... And, like, everyone is, like... Because he created a clock code, it's, treating his word as gospel. Totally. And then, no. Well, sometimes he disagrees himself.
00:21:01 · Shawn Wang, Mihail Eric
[00:21:01] S0: And there's people who went very, very far at Dog Factory and then came out saying, was wrong, guys. So we covered that on AIE. Just be aware that not everyone... They probably believe the things they're saying at the time, but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, yeah, I am often wrong. Yeah. Like... And, like, everyone is, like... Because he created a clock code, it's, treating his word as gospel. Totally. And then, no. Well, sometimes he disagrees himself. [00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap.
00:21:29 · Mihail Eric, Shawn Wang
[00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap. [00:22:09] S0: Yeah. I will call out... So some of your... For example, you have Cognition on the week that you're talking about code review. Yeah. I actually did work with Silas on dev and review. [00:22:18] S1: Yeah. Yeah. [00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use
00:22:19 · Shawn Wang, Mihail Eric
[00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use [00:22:29] S1: you guys. That's also something like, yeah. So Monaco does use Devin. And that's... Mean, and Silas actually spoke last year, and so Silas is great. It's just super, super smart. But yeah, we use you guys, and so I was like, okay, among the providers of kind of code review tools that we... And we sampled a lot of them. We actually did kind of converge on Devin for a lot of use cases. [00:22:47] S0: Silas actually more on the model side these days, and then actually it's Walden that you want. [00:22:51] S1: Yeah. Yeah. Yeah. I've spoken to Walden as well. Yeah. [00:22:53] S0: Yeah. I will also shout out Eno from Factory because, you know, talking about agent readiness, they've been all about that. They build signals, tooling on that. Do you guys use that? [00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah.
00:23:02 · Mihail Eric, Shawn Wang
[00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah. [00:23:27] S0: It was fine. I I was at a call. We were talking about this stuff, and, like, we were like, okay. We need to form some opinions about it. And then, like, factory, like, shipped a full product on it that week they were talking about it. I was like, Okay, you won. I mean, everyone kind of agrees on the same things. [00:23:41] S1: Sure, sure. [00:23:41] S0: And it's just a question of, Do you want to turn it into a dashboard? Yeah, yeah, yeah, yeah. Because it's dashboard. [00:23:46] S1: Right, right, right, right, right, right. But you know, value. And I think they did well from a marketing perspective, jumping on the ratings report. And I've definitely spoken to... I will regularly pitch that to other people and say, yeah, you should run ratings reports of your own kind. Of course, now people have open source versions of it, I think the principles are super important.