Latent Space: The AI Engineer Podcast
01:35:49 · Mihail Eric, Shawn Wang
[01:35:49] S1: That I will give you. That I will give you. I can I can accept that? That actually, you can can agree with. [01:35:53] S0: Yeah. And and then then, by the way, it also folds into the other thing of, what did I say? Was off z, you know, permissioning and all these things and billing. So, like, to the limit, all infra converges to the same exact problems where they have the box in the middle and then you have the box in two sides. The, you know, godbox will solve everything. Right? And For the enterprise. True only if you take... If you understand the job of the godbox. And so separating MCP portals and gateways means you don't fully accept that they are, you know, the the the same layer of problem. In a very similar way, I also find that people don't understand database querying to have a lot of the similar things where they will separate filtering from ranking from... Or or just, like, general recommendation Mhmm. If they're the same problem. So then you just come... Because when you take all these things, you combine them in the same layer, then you can sort of cross depend on things in useful ways. If there are different layers, then they cannot.
01:35:53 · Shawn Wang, Mihail Eric
[01:35:53] S0: Yeah. And and then then, by the way, it also folds into the other thing of, what did I say? Was off z, you know, permissioning and all these things and billing. So, like, to the limit, all infra converges to the same exact problems where they have the box in the middle and then you have the box in two sides. The, you know, godbox will solve everything. Right? And For the enterprise. True only if you take... If you understand the job of the godbox. And so separating MCP portals and gateways means you don't fully accept that they are, you know, the the the same layer of problem. In a very similar way, I also find that people don't understand database querying to have a lot of the similar things where they will separate filtering from ranking from... Or or just, like, general recommendation Mhmm. If they're the same problem. So then you just come... Because when you take all these things, you combine them in the same layer, then you can sort of cross depend on things in useful ways. If there are different layers, then they cannot. [01:36:52] S1: Yeah. I actually will not disagree with this. I think maybe the... Yeah, so it's slightly pedagogical. I think that this breaking up of separate terms versus one, I think it is absolutely correct that the underlying architecture is the godbox, where one thing goes in and there's a bunch of other stuff that is some third party, some internal... Some first party that all kind of exists on the other side of the godbox. So, yes, I agree with this, actually.
01:36:52 · Mihail Eric, Shawn Wang
[01:36:52] S1: Yeah. I actually will not disagree with this. I think maybe the... Yeah, so it's slightly pedagogical. I think that this breaking up of separate terms versus one, I think it is absolutely correct that the underlying architecture is the godbox, where one thing goes in and there's a bunch of other stuff that is some third party, some internal... Some first party that all kind of exists on the other side of the godbox. So, yes, I agree with this, actually. [01:37:18] S0: I I have a I have a spicy spicy tweet. I... Oh, god. Where where is the godbox tweet? Oh, yeah. So this is the m m m x Sure. N Sure. M plus n Let me bring it up. Architecture. So anyone working in infra needs to understand. Okay. Congratulations. You now have a job in infrastructure or you want to be an infra engineer. You need to understand that your entire job is making god boxes and then taking them apart. So either... Your problem is either you don't have enough... You don't have a good god box, or your problem is someone else's god box you don't like, so now you have to take it apart and make your own. [01:37:51] S1: That's funny. That's very funny. I like that. That's very funny. Into the heat death of the universe. Wow.
01:37:51 · Mihail Eric, Shawn Wang, S2
[01:37:51] S1: That's funny. That's very funny. I like that. That's very funny. Into the heat death of the universe. Wow. [01:37:58] S0: That's funny. Which is a... This is a restatement of, like, there's only two ways to make software, which is you bundle them, you unbundle them. Right. Right. What are you bundling? Is It the things that go together, hopefully. And then maybe over time, people's opinions shift as what goes together, what should not go together. So there is that whole thing, maybe Monaco should be more of a point solution. Maybe Monaco should own everything. Right? And, like, there's the question of, like, in some ways, you're bundling for the the kind of startup segment that you serve. In some ways, you're unbundling from Salesforce because Salesforce... And then that's how you win market share from Salesforce. You unbundle from Salesforce. Mhmm. Mhmm. [01:38:39] S1: That's interesting. Yeah. That's that's an even broader statement. I mean, connecting it to broadly all of vertical AI or... Right now, like, bundling, unbundling. I mean, that's like the whole, like, Craigslist and Airbnb, you know, the the Tinder and all those are separate. Tinder. But, you know, like, a personals, basically. Yeah. Yeah. Yeah. Yeah. [01:38:57] S0: Yeah. Never thought it... Never thought thought about it that way. But, yes. And, you know, one one of the things that... So we just shipped the OpenRouter Pod today. Right? And OpenRouter, huge success. [01:39:06] S2: Mhmm.
01:39:06 · S2, Shawn Wang
[01:39:06] S2: Mhmm. [01:39:07] S0: Lots of people tried to build gateways. None of them had success with OpenRouter. OpenRouter only builds gateways. So there's an example of, like, this... They disagree with me that the the the router layer should also do ops because Arize is doing ops and observability and and what have you. And and they... And, like, the MCP gateway, they don't have that. And they they they just... They only handle model routing. And so, like, I I, like, I am... I love being wrong like that because they're like, we are the point solution, and we only do this one thing really, really well. And that's our that's our engineering focus, which is good, but it's also the marketing focus, which is nice. Mhmm.
01:39:43 · Mihail Eric
[01:39:43] S1: I I think the... This becomes... I think that the... What you've described of, like, the one one stop shop for all the things is ideologically what makes sense, but I think what ends up happening in practice, and you know, part of it is probably also marketing, is that, especially when you look at the enterprise, is like this is broadly under infra, like broadly under platform, but then there's separate teams owning different parts of the stack. And I think part of that consolidation at the at the scale of an enterprise, if we're doing MCP portal and gateway, someone has to come together and say, like, now we're... There's a single endpoint that you're hitting for both of these things. And that fundamentally is, like, a hard thing to, you know, get everyone on board with. So oftentimes it's more political and even people problem rather than from a pure software perspective, this makes a ton of sense. But that's not it. Software systems sometimes are bottlenecked by people problems more than pure software.