Latent Space: The AI Engineer Podcast
01:09:33 · Shawn Wang, Mihail Eric
[01:09:33] S0: Yeah. For for people who I have never worked at this scale, because it it took me a while to to to see this, it is on the order of, let's let's call it tens of thousands of repos, more repos than developers. These are the situations where, like, you don't know what code is running That's absolutely anywhere in the company because probably the guy I wrote is no longer here. Right? That's absurd. By the way, it's still load bearing. [01:09:55] S1: That's absurd. Yeah, that's the scariest thing. That's absolutely right. And then you're the person who's on call, and you're interacting with a system that goes down where that person still isn't there, and there's like old out of date runbooks, you're like, what the hell is going on here? It's a scary, scary thing. And yes, at the enterprise level, it's a whole different beast. [01:10:16] S0: Yeah. So kind of college hacker, one person per repo, they've never... Totally. No appreciation. But I think if you just give that idea, you drop them in a simulation of, okay, day one on your job, this happens, what now?
01:10:16 · Shawn Wang, Mihail Eric, S2
[01:10:16] S0: Yeah. So kind of college hacker, one person per repo, they've never... Totally. No appreciation. But I think if you just give that idea, you drop them in a simulation of, okay, day one on your job, this happens, what now? [01:10:32] S1: Yeah. Yeah. So another shout out I'll give here. So the Resolve folks, actually last year, they made an assignment for the class where they effectively created a sandbox for students to go and then the sandbox was like strewn... I mean, within the sandbox, there was, like, many many services that were doing their various things, and there was even, like, bugs in the services within that sandbox. And then as part of the assignment, the students had to, like, go and actually, like, use the Resolve agent and kind of, like, probe and play the the human on call person probing the system, discovering what's happening. [01:11:07] S2: They've gotten interesting. [01:11:08] S0: Yeah. Yeah. It's an interesting website. [01:11:09] S1: I do not. It's been a while Yeah. Since I've seen [01:11:12] S0: This is very interactive. Okay. They're being a PagerDuty now. [01:11:17] S1: Yeah. Checks But also this, like, this video, I was... [01:11:21] S2: You know, you were shouting out. Interesting. Interesting. But okay.
01:11:21 · S2, Shawn Wang, Mihail Eric
[01:11:21] S2: You know, you were shouting out. Interesting. Interesting. But okay. [01:11:25] S0: Yeah. Is it... So... Okay. We, you know, we never get a chance to talk about this side of the world except for conversations like this. Any other segments? Because I I think AI SRE is maybe too broad or even too narrow, depending how you look at it. Right? Like, too broad meaning, well, there's there's several subdivisions of AI. Sorry. That's interesting. Mhmm. Or too narrow being like, well, actually, that's not even the biggest problem that we have. Mhmm. So I take it however you like. [01:11:48] S1: Yeah. You mean, like, specifically for for, like, Monaco or just, like, broadly for, like, organizations thinking about this? [01:11:53] S0: Broadly, because that is our audience.
01:11:53 · Shawn Wang, Mihail Eric
[01:11:53] S0: Broadly, because that is our audience. [01:11:56] S1: Mhmm. Take a look at that. Yeah. I think that, again, it's function of how we just scale. Right? And I think that for the enterprise, there are a number of issues that aren't always I almost feel like the answer is not even always just more agents, and and since we're still actually on the week five agent ready code bases, like, I talk to companies of, like, a certain scale, the the things that are holding them back are really more just, like, low level DevOps, CICD stuff, right? Their merge queues are getting backed up. It takes too long to stand up an app. There's just actually more foundational things that have almost nothing to do with, that throwing in one more SRE bot, or a la a bot wouldn't be the answer to the problem, because there's more systemic things that they need to figure out. And so... So that brings us to [01:12:38] S0: the week six.
01:12:39 · Mihail Eric
[01:12:39] S1: Yeah. Exactly right. So that's where we get into... Yeah. Exactly right. Yeah. There we go. So the MergeQ and beyond. Yeah. This is a love letter of sorts to modern, gentic code review. Yeah, I think that biggest thing that maybe happened between last year and this year was just it suddenly became very clear that code review is a bottleneck, especially once you people got really, really, appealed and the velocity on generation became so high that, you know, you could, in theory, like, put up 30 plus PRs a day, and then what? Like, what's... At what point do you decide that you either have to institute yourself and put yourself in the middle as some way gate and review things in the traditional way, but then you're literally the constraint, you're the choke point of the velocity, and that's not good. Right? And then what was the point of generating at scale? Or you say, well, how do we maybe get the best of both worlds? Where we're instituting ourselves enough that we have trust in what's getting merged, but we're also able to kind of merge at scale, but we're also not kind of freewheeling it here, and anything goes. So a lot of, I think, what I wanna touch on here is just that middle ground, right? Which is like, there are... There is a class of problems that if you've put in thoughtful design into your agent readiness, you've put in thoughtful design around your verifiability, there's a class of problems that you can feel comfortable, you know, some percentage of the time, essentially, like auto approving. And what has to be true from a kind of a design perspective, from like an architecture perspective, to just auto approve? And when do you have to insert yourself and say, actually, here, I still wanna review this carefully? And even if you do review it carefully, are you still doing line by line review, are you doing something else? Right? So yeah, some of this.
01:14:13 · Shawn Wang, Mihail Eric
[01:14:13] S0: One of our hottest topics of this year, we have a guest writer program for Latent Space. This is from Ankit from Aviator. [01:14:20] S1: Nice. [01:14:20] S0: And he has a whole thing on killing the code review, because I... You know, that is, like, maybe the final boss Yeah. Of of the... That's that's a bottleneck. He has a few, gates by which he does this. I don't know if you... You've talked to Funky or... [01:14:34] S1: Yeah. No. I've I've read this post. Yeah. I haven't spoken to him personally, but I've I've definitely... [01:14:38] S0: Mostly right. Any criticisms? [01:14:40] S1: No. I think this is right. [01:14:40] S0: I mean, [01:14:41] S1: like, broadly right. I guess you mentioned one thing, like, last boss. Maybe not... I I think there's two choke points. There's the review, and there's probably, like, the intent or, like, kind of the thing at the top. Right? Like, [01:14:49] S0: is the And there's some there's some stats as well that that I like to pull up. Everyone's stats is different. [01:14:54] S1: For sure, for sure. [01:14:54] S0: But clearly, more people are merging. Clearly, the number of reviews are not increasing in line with merge Yeah. That's