Latent Space: The AI Engineer Podcast
01:15:03 · Mihail Eric, Shawn Wang
[01:15:03] S1: right. That's And if you look at any modern sort of company that's really invested in this and thought about this carefully, and some people have already done posts about this, and I'm sure we'll do posts about this as well. But I think recently, like, the Amplitudes and, I think Cloudflare has has done a good one. I think Augment has also done a good one, where they talk about the design of their AI code review system. These are a bunch of really good, if you wanna share some of those links. Think they're fantastically written posts. Yeah. So shout out to... I actually, shout out to Cloudflare, because I think, like, very, like, kind of, like, know, in a sense, like, legacy company, or, like, you know, bigger company. But I think they've done a fantastic job in terms of just their their content around the AI, the AIS DLC. So I think they have a good post there. But TLDR, yes, I think this is true. I think that there's... We're still finding that sweet spot. What is auto approvable? What is not really auto approvable? And if it is not auto approvable, is even in that world of something that has to be somewhat human reviewed, what do we review? Are we still doing the line by line? Are we doing some combination of the spec? And, you know, I think this... Yeah. Again, it's a whole separate question there as well. But... [01:16:09] S0: Are are people keeping, are you guys keeping the spec up to date at all times? The spec is no way. Right?
01:16:09 · Shawn Wang, Mihail Eric
[01:16:09] S0: Are are people keeping, are you guys keeping the spec up to date at all times? The spec is no way. Right? [01:16:16] S1: Yeah. No. Well, I mean, okay. In a few different places, right? So, like, there's a spec that as it exists in, like, a linear issue, and there's a spec as it exists in, like, the context of a PR description, right? So you can, like, you can have your PR description be the place where you, like, describe more or less the entirety of the implementation in a more condensed format. And as long as that's the most up to date thing representing this point in time of the code base, I think that's sufficient, actually. So we're thoughtful about our peer descriptions. We make them be not just, like, you know, a one paragraph. It's more like, it is effectively like a spec of what has happened in the PR, and there's even, like, kind of line file level pointers to things where things have changed, so you not only get the high level of what's happened, but even the kind of the construction of the files and sort of the service level boundaries that you're touching. So yeah, I think that's where we've invested quite a bit.
01:17:00 · Shawn Wang, Mihail Eric
[01:17:00] S0: So I push back on that because to me, of course, it's, you know, bare minimum, it should be internally consistent with the PR spec. Sure. Yeah. No. What I meant is the spec for the feature, the code base, because that is the submission of all PRs. That is like, if your if your PR spec got it wrong or missed something, it's gonna be the delta between what the whole code base or the whole history of the code base is supposed to do versus what your PR is trying to do. Mhmm. Yeah. So so I'd like... I mean, I don't I expect you to have an answer. Yeah. Because, like, I I really stress about this myself. I try to instruct models to do it. They're bad at it. That's it. So so the are the the spec gods of, like, well, five weeks ago, did this other thing, and now you're reversing it. [01:17:42] S1: Have you seen the post from Basis? Like, they have a... They had a tweet little thing, a tweet article. I thought they did a great job talking about, like, different sort of file artifacts they they... I just tracked via git, and some of them are, like, the specs, the intent of what is to be, and the other is, like, what they call, like, the canonical state of the code, which is, like, a little bit more maybe what you're describing, which is implicit knowledge, implicit intent, that also has to be codified somewhere, right? So that part, that's all version controlled. And I think they found that to work for them, so that might be another thing [01:18:12] S0: to share Yeah, as Vivo was trying to pull it up. This is based on the accounting company.
01:18:12 · Shawn Wang, Mihail Eric
[01:18:12] S0: to share Yeah, as Vivo was trying to pull it up. This is based on the accounting company. [01:18:15] S1: The accounting company, exactly right. [01:18:16] S0: Yeah, yeah. We're having Mitchell at my New York conference soon. Amazing. Talk to him about that. I like that they are coming out with this. My solution for this is, I don't know if like how ready I am to talk about it, because I may never ship it, is something that is replacing Git in order to capture all of this. So I call it a program database where it captures all traces and all... And it reifies specs to something that that we that we can ignore and and keep, and it also controls the skills that are attached to it as well. [01:18:49] S1: That's [01:18:49] S0: cool. So I think like, basically what Git... Git is only focused on code, and then you just need a source of truth that is not only everything else outside the code, including the the implicit... What what did call it? [01:19:00] S1: Implicit assumptions are like the constraints, the intent, the other stuff. [01:19:03] S0: But also, highly concurrent. Because Git, by design, introduces mutexes everywhere, which sucks. So you need super concurrency. [01:19:16] S1: Have you seen... Is it GitButler? Is it? Yes. [01:19:20] S0: I'm the guy who [01:19:21] S1: used to be CTO. Ian, [01:19:23] S0: I think is his name. [01:19:23] S1: I forget his name, but yeah.
01:19:23 · Mihail Eric, Shawn Wang
[01:19:23] S1: I forget his name, but yeah. [01:19:24] S0: Scott. Scott. So I think there's a lot of people going after this, and I think there's the question of, like, really what is the ideal representation of tasks? And I think the winner, probably not me, linear. [01:19:40] S1: Really? [01:19:41] S0: Because that's where [01:19:43] S1: Yeah, yeah. The tasks [01:19:45] S0: clearly they have decided to become a coding agent company. And well, they have all the sorts of truth there as long as you buy into linear. [01:19:55] S1: Yeah. Yeah. That that is true. That is true. I I wonder... I I have, like, this somewhat, like, kind of held belief that I think a lot of companies are gonna evolve into some version of a coding agent company, which is any company that has, like, an interaction with b to b software where you're collecting something about the user or some, like, product insight, I think you can also close the loop. Because ultimately, like, a coding agent is... The purpose of coding agent is to act on some adventure thing that's happening, where a user is involved or a customer is involved, and they use that to, like, fix, ideally, aspect of the product. Right? I think what Linear is doing, I imagine the same thing like Amplitude could do as well. Right? Because they're also getting insights about user analytics, user problems. I imagine even, like, you wanna really stretch it. You could even have, like, Decagon or CRM, do something similar, where, you know, here's what's broken. Go close the loop. Mhmm.
01:19:55 · Mihail Eric, Shawn Wang
[01:19:55] S1: Yeah. Yeah. That that is true. That is true. I I wonder... I I have, like, this somewhat, like, kind of held belief that I think a lot of companies are gonna evolve into some version of a coding agent company, which is any company that has, like, an interaction with b to b software where you're collecting something about the user or some, like, product insight, I think you can also close the loop. Because ultimately, like, a coding agent is... The purpose of coding agent is to act on some adventure thing that's happening, where a user is involved or a customer is involved, and they use that to, like, fix, ideally, aspect of the product. Right? I think what Linear is doing, I imagine the same thing like Amplitude could do as well. Right? Because they're also getting insights about user analytics, user problems. I imagine even, like, you wanna really stretch it. You could even have, like, Decagon or CRM, do something similar, where, you know, here's what's broken. Go close the loop. Mhmm. [01:20:37] S0: Yep. Alright. So set all those things up for agent ready code base and set up code review. Security is week seven. It's not week one. Students. What do you mean security is not important? No. I make fun of because I always say, like, you know, at Amazon, job zero is security, and they always say this. Yes. But... Yeah. Security review at the end. [01:20:56] S1: Okay. I was at Amazon, and I don't remember that. But I I believe This is [01:20:59] S0: the thing Andy Jassy says.