Latent Space: The AI Engineer Podcast
01:03:38 · Mihail Eric, Shawn Wang, S2
And it it was picked organically. Like that's... [01:03:40] S0: So purely vibes. Right? Like, this is... There's no way to even... [01:03:42] S1: There was... Yeah. This this was like... I I mean, okay. Okay. Here... Here's what I'll say. When we did that first evaluation the first time around, where we didn't really pick any vendor, and we were like, there's no vendor that we're super happy with, I did do, like, an analysis where I looked at our PRs, basically, it's sort like a, you know, set of pie diagrams, like, here's what this is solving. Are we we happy with the of the trade off of verbosity and how often it's posting with utility? And so there was... It was more statistically driven than just vibes. The second time around, I wasn't gonna push hard on it. The first time, I was kinda like, hey, we should adopt an AI code tool. Second time, I was like, no, I'm not gonna do that again. But then organically, people were just like, Devin is, like, good. Like, can can you, like, add me to this? And so today, people are sort just, like, getting organically added. So this is a vendor where, you know, one of the cases where we're still leaning to the vendor. You know, we'll see in the long term what that looks like, but everything outside of that, we just didn't feel like maturity was there. [01:04:30] S0: Yeah. And it's not painful enough that you would, like, rip out, rip out Devin to write you on. [01:04:34] S1: That's right. [01:04:35] S0: That's interesting. [01:04:35] S1: Yeah. It's... [01:04:36] S2: It's a good good talk for Devin.
01:04:36 · S2, Shawn Wang, Mihail Eric
[01:04:36] S2: It's a good good talk for Devin. [01:04:41] S0: Even know, you know, for the record, I'm only an advisor. I'm not that involved anymore, but, like, it's interesting to to hear about it externally. Mhmm. Because internally, there is a certain amount of, let's call it, the bubble, where on Slack, you only see your positive things, because people only want to circulate positive things. But you have no dog in the fight. You just choose the [01:05:03] S1: best tool. I mean, if you want to get us free credit, totally... That'd helpful too. Say saying [01:05:08] S0: no no I to think Silas will hook you up. Okay. So there's that, right? And that informs all your teaching work. [01:05:17] S1: Yes. That's right. [01:05:17] S0: That's right. I think that's just really good. And we can bring in more of this as you like, because I think this lived experience is very good. Yeah. I'm happy to go back to agent ready code bases. So week four and five kind of [01:05:32] S1: matching Yeah, they blend a little bit. Yeah, that's right. [01:05:36] S0: Whatever you want to, whatever strong beliefs you have introduced in Monaco [01:05:40] S1: Yeah. [01:05:40] S0: And recommend people to start with.
01:05:42 · Mihail Eric
[01:05:42] S1: Yeah. No. For sure. Yeah. Week four and five, I would say they they are a little bit... Four is maybe a little bit more, like, conceptual foundations and ways to do this, because there are a lot of ways to customize agents. And then I would say week five is where I really kinda, like, lean into, alright, let's think about this and apply this in practice on, like, a true code base. And so here, I will definitely, you know, give a shout out to to the factory team and and, and wanna just highlight the value of something like a readiness report. I I think that the main piece that I want to reinforce, something like agent ready code code bases is to me like a precursor and a bit of a building block toward everything to come afterwards. When I say everything to come afterwards, I mean, like, people always got, you know, for a while, and I think people still are, like, super hyped on, like, multi agents, like, paralyzing agents, like, tens of agents doing all kinds of things. It's like you actually need to set a foundation in order to do that well, and the foundation is how verifiable is your code base, how, like, well structured is your code base, how understandable is it by an agent. And so I think, like, agent readiness to me is, like, almost almost like piece one of the next four to five weeks where... Maybe until, like, week eight or nine where where I spend more time talking about, like, broadly, you know, AI software teams and, like, kind of org level structure. But definitely building up the building blocks for background agents of the software factory. I think this is where it all kind of gets interesting from that perspective.
01:06:59 · Shawn Wang, Mihail Eric
[01:06:59] S0: Yeah. One thing that I observed you talking about from Monaco, but it's not here, is the SRE side. When you close the loop, you actually want to have the agent read your logs for builds, and then ideally read your analytics for live production. [01:07:15] S1: That's right, that's right, yeah. Not explicitly, but it's a really good shout out. So I think that I'm gonna weave that in in like two places. One is in the background agents, week, and then also in the software factory week. So you're absolutely right. I do... It's a really good call out. This idea of, like, trigger slash event based closing of the loop, which, you know, now is one of the one of the two or three entry points when people talk about agents kicking off tasks in the software factory, that's absolutely right. That'll come in there. [01:07:40] S0: Yeah. It's interesting that people call it trigger or event based, which I guess that is the actual technical implementation of it. Yeah. It just seems too low level almost. Mhmm. But I... So so the trigger is usually a cron job. Right? Like, let's let's be real. [01:07:55] S1: Sure. Sure. [01:07:55] S0: Sure. Event based is usually a cron job that, like, surfaces an event, and then it's event based. But it's all crons.
01:07:55 · Shawn Wang, Mihail Eric
[01:07:55] S0: Sure. Event based is usually a cron job that, like, surfaces an event, and then it's event based. But it's all crons. [01:08:02] S1: Yeah. Yeah. That's right. That's right. Yeah. Yeah. Or, yeah, like, webhooks or something like that. Yeah, that's absolutely right. Yes. And, you know, the technical term, of course, because at the end of the day, yeah, being sort of like a a person by by by trade and and, yeah, we we we call those things, and so I guess that's... I fall back on that terminology. [01:08:19] S0: Yeah. And have any AI SRE companies made an impression on you?
01:08:19 · Shawn Wang, Mihail Eric
[01:08:19] S0: Yeah. And have any AI SRE companies made an impression on you? [01:08:25] S1: So, okay. I mean, I think the Resolve folks are doing well there. Resolve is doing well. I would say Nightshift is also doing quite a good job there, in terms of broadly background agents. I'm a big fan of kind of... They're Greylock backed. I think they're doing doing real well in this area as well. Yeah. I I would say that where... I think SRE is an interesting use case, and this might... Back to the build versus buy question. I think what's interesting about it is, like, where do... When does it become... Like, where does an SRE, an AI SRE, like, really see the most value? It's typically when you have, like, a tremendous number of, like, service dependencies, and these, like, massive sort of, systems, and tons of Kubernetes and Helm files, like, you know, like, tremendously large Kubernetes infrastructure Helm that's, like, files kind of strewn everywhere around. I think that's why Resolve has done really well in, like, the enterprise, right, where you go and you sell to a Fort, you go and sell to, like, a Walmart, and you kind of say, hey, you have... Your service level dependencies are crazy. And you see these these, like, you know, these diagrams of how things are set up. Then And they kinda go in and they're like, we'll we'll we'll try and make sense of all of this. I think that, like, at at smaller scales, it's usually still manageable that you can kinda roll your own. But, you know, I will say that for the enterprise, think they've they've done a good job.