Latent Space: The AI Engineer Podcast
00:17:26 · Mihail Eric
So how do we unblock that or unbottle the the bottleneck a little bit? What mechanisms are there there? And so verifiability and, you know, things of that nature and how we can extend that into the code review process. Week seven, security. I mean, obviously, I think there was a point... So this was in last year's class as well, and this this I wanted to reinforce. I mean, with any autonomous system taking a lot of actions, we're seeing a lot of... This year, especially, a lot of things getting hacked, you know, and not least of which are are just some of the the core pieces of software from from Vercel to, you know, gateways, LM gateways entirely. So I wanna spend some time just, like, talking about what new attack vectors exist exist in security and and, you know, how we can mitigate some of them. Week eight is background agents. So this is, what I wanna start touching on here is, you know, how do you, like, basically run agents in many different, like, product service areas, not just in what was traditionally an IDE, but also having things that are, like, based on triggers in your code base, in in your repo, you know, during CICD, things that get triggered even when, you know, based on events, things like SRE style bots. How do you even build a system like a... I think this past year was when a lot of companies started releasing their posts like Ramp Inspect. There was... Spotify has their own version of this. You know, I think Stripe Minions, so all these folks were doing really cool write ups of, like, their internal info they were building to do this. I wanna touch on that, get people exposed to what that is. Yeah. This is, yeah, this one is more on, like, what happens when you're in a team in in the modern world and sort of, like, the core pieces of of team level infrastructure. So everything from, LM gateways to, you know, MCP portals, which is, like, MCP at scale. And then just also how to think about the evolving roles, I would say. So this is maybe a little bit like a soft thing. It's like, what is now... And the vanilla, maybe not super nuanced thing to say is like, everyone's a builder, but I wanna go a little bit deeper beyond that and sort of like, what thing is actually getting subsumed? What roles are actually either evolving or merging that didn't once exist?
00:19:28 · Mihail Eric, Shawn Wang
What should students think about in terms of defensible skill set that maybe wasn't true before? And so a lot more team dynamic type considerations. And then, yeah, this is a... It all kind of builds up toward the toward the factory, which I think is top of mind for now, for a lot of folks today. Whence we're we're building factories, where do humans still exist and all that? What is the... You know, where are the new choke points that that are intentionally there around around human involvement? Things like intent, things like setting up a verifiable systems, and what happens in the limit when maybe 89% of code is fully driven by just agents, where we kind of filled into all that. So, yeah. [00:20:04] S0: Nice solid 10. I was gonna say, like, you know, some of these are more established patterns than others. And I think people need to know that not all of this is settled science. [00:20:18] S1: True. [00:20:18] S0: You know, but it is everything... I'll endorse everything as things that people talk about and you should know. You just... We know the open problems, we may not know the answers.
00:20:18 · Shawn Wang, Mihail Eric
[00:20:18] S0: You know, but it is everything... I'll endorse everything as things that people talk about and you should know. You just... We know the open problems, we may not know the answers. [00:20:27] S1: That's absolutely right. Yeah. Yeah. I mean, even... I mean, you probably have your own version of this. I'd be curious to hear what you think is still defensible, or not defensible, but not as set in stone. But, you know, even like the the softer factory. Are we talking about, like, dark factories? Are we talking about, like, these sort of half lights on factories? Like, I think even there's a spectrum there. Yeah. And there, you know, there's the people that are on the on the very frontier, like like the warps and the factories that are that are, like, building factories and selling them the services. And, you know, even they will only claim, like, 30 some percent, if even that, of of, like, fully, you know, identically driven contributions. So Yeah. What does that mean in the limit? It's... It is hard to say. [00:21:01] S0: And there's people who went very, very far at Dog Factory and then came out saying, was wrong, guys. So we covered that on AIE. Just be aware that not everyone... They probably believe the things they're saying at the time, but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, yeah, I am often wrong. Yeah. Like... And, like, everyone is, like... Because he created a clock code, it's, treating his word as gospel. Totally. And then, no. Well, sometimes he disagrees himself.
00:21:01 · Shawn Wang, Mihail Eric
[00:21:01] S0: And there's people who went very, very far at Dog Factory and then came out saying, was wrong, guys. So we covered that on AIE. Just be aware that not everyone... They probably believe the things they're saying at the time, but they have the right to change their mind, and they do. And even Boris recently tweeted that, like, yeah, I am often wrong. Yeah. Like... And, like, everyone is, like... Because he created a clock code, it's, treating his word as gospel. Totally. And then, no. Well, sometimes he disagrees himself. [00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap.
00:21:29 · Mihail Eric, Shawn Wang
[00:21:29] S1: Yeah. Yeah. And that's okay. I mean, that's, like, this isn't politics. Right? This is... We're we're still scientists. We're engineers. We're we're subject to, I don't claim to be dogmatic, but, you know, I just want people to experiment. I think that's, like, a principle as well. You asked what principles earlier. Just like experimentation. I mean, like, just have to be ready to... And oftentimes, when I when I sort of, like, give sort of these lectures or or talk to people about this stuff, they're like, how do I... There's so much going on. How do I stay on top of this? And it's like, you know, there's something simple just like experiment for ten minutes a day. Like, just spend some ten minutes doing something that you didn't do before. If that's a traditional way that you were solving a problem, there's now a new tool that you wanna try or a new framework or something, and just spending, maybe even doing the exact same problem just with that new kind of thing alone will cover most of the gap. [00:22:09] S0: Yeah. I will call out... So some of your... For example, you have Cognition on the week that you're talking about code review. Yeah. I actually did work with Silas on dev and review. [00:22:18] S1: Yeah. Yeah. [00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use
00:22:19 · Shawn Wang, Mihail Eric
[00:22:19] S0: Pretty good. I would say, like, you know, it's hard to benchmark these things. Sure. There is a review bench. No one uses it. Qualitatively, the review is pretty good. Well, we use [00:22:29] S1: you guys. That's also something like, yeah. So Monaco does use Devin. And that's... Mean, and Silas actually spoke last year, and so Silas is great. It's just super, super smart. But yeah, we use you guys, and so I was like, okay, among the providers of kind of code review tools that we... And we sampled a lot of them. We actually did kind of converge on Devin for a lot of use cases. [00:22:47] S0: Silas actually more on the model side these days, and then actually it's Walden that you want. [00:22:51] S1: Yeah. Yeah. Yeah. I've spoken to Walden as well. Yeah. [00:22:53] S0: Yeah. I will also shout out Eno from Factory because, you know, talking about agent readiness, they've been all about that. They build signals, tooling on that. Do you guys use that? [00:23:02] S1: Yeah. Indians? Yeah. So so we do use agent readiness of our own of our own kind. I mean, Eno is is awesome as well. Like, I I think they were super early on this on this train. And I think when he codified that in his talk, I guess, at AIE, what was that, like, last... I forget when it was. But one of his yeah. Yeah. Europer. Just fantastic talk. And I think it was kind of, in a way, of its time. And so I was like, there's only one person that I could see being, like, the most appropriate person to talk about this, where I, really wanted to give them the spotlight, they're super excited. Yeah.