The MAD Podcast: How AI Gets Built — with Matt Turck
00:32:28 · George Sivulka
[00:32:28] S0: Yeah. I think that some of the most interesting stuff at Hevia isn't actually only the agents research, but actually the cure systems problems where running... I think we run around two fifty billion large language model calls a month. And our workflows, I think before we had Maximizer with all the rate limits that we had, we could run a million tokens a minute, and now we can do 500 or four fifty million tokens a minute. All of that actually wasn't an AI problem, it was a systems problem. And like you mentioned, it is a router. You could think of it as a two sided marketplace, but one of our systems there is called Maximizer. We liken it from an analogous perspective to an air traffic controller, where just like an air traffic controller basically tells you who can fly where and when. And apparently you're not supposed to fly out of Newark right now. I hope it's fixed. Maximizer basically has a handshake between license grants and license requests. So if a user or a job has a really high or low priority and wants to go and use GPT-four 0.0, it can make a license request and license grant will say, okay, you have to use Azure or you have to use X, Y, or Z models. And it actually ends up with the theoretic, the information theoretic maximum utilization of any rate limits. So not only are we running the most amounts of pages, the most amounts of tokens through our platform, but we're doing that in the most efficient, perfect way, and that's all our infrastructure, which is a cool problem. I could get into more, but yeah, I don't want to bore it around.
00:34:10 · Matt Turck
[00:34:10] S1: Inevitable question about hallucinations, you know, high stakes professional context where people have paid a lot of money to provide very reliable results to the customers. Is that something that, you know, in 2025 is as much of a problem as it was?
00:34:26 · George Sivulka
[00:34:26] S0: I think it's old news. And I think the only reason we still talk about it, it's like everyone talks about hallucinations, and, you know, no one knows if it's happening or what's going on. It's like, it was a problem back when the models were stupid, but they're obviously not stupid now. And I'd even say that they're way better than any human. So, when we start to think about like hallucinations, it's like, well, where does that come up? Like, have we actually seen a lot of that happen? Probably not recently. But the place where it does come up is actually a limitation of rec. It's when you've got the wrong documents or you're searching for something and it finds something that's broken. And what you're realizing is that the models are actually pretty good at doing reasoning. They're really good meta learners. Like the hallucination, whether it's a problem or not, no one really cares. What people care about is when these things fail because they don't have the right context or they're not reading the documents in the right way. And that's why we've done a lot of research on how to feed the right stuff at the right time. And if that's really, really expensive, we don't care because models are gonna become... Intelligence will become too cheap to meter. That's what our industry is predicated upon. So, we'll spend $10,000 in model costs per user a year. We wouldn't care. We don't, but indicative.
00:35:47 · Matt Turck
[00:35:47] S1: You just mentioned research. How does that work at at FBI? Do you guys have a concept of an AI lab within the company, or is it distributed across the team? And how important is research, especially in a context where, again, there seems to be something new every other day? How do you think about doing your own research versus just free writing whatever comes?
00:36:11 · George Sivulka
[00:36:11] S0: Yeah. I... It feels like there's something new every day. But also, like, if you've been tracking the field, it also doesn't. Like, it it kinda doesn't feel that different. I don't know if I'm allowed to say that, but it doesn't for me. There's new models, but they don't feel that different. Some of them are more sycophantic. That's the news. I think, and not a lot of people are saying it, but I think SF has a really big group think where it's like, okay, everyone's gonna go out and work on the same thing. There's this sort of memetic attraction rivalry between OpenAI and Anthropic. And what are they gonna do next? And like, oh, they have a new interface. So we're like, the chat bar got a little bit jumpier with the animation. And we kinda know, okay, they can make videos and it's too slow, and they can make images and it's better now. The issue is to come up with new interfaces and AI that can really do new things, I think you kind of have to avoid the group think. And so we currently don't have any AI researchers hired out in San Francisco. We don't even listen a lot to what people are requesting and really try to think about, okay, what is the tool that AGI would use? Like if you actually had AI that could choose any other AI tool to use, what would that look like? And we have that guide, what the interfaces look like. That's kind of how we came up with Matrix and a lot of the other interfaces that we're talking about. That's how we came up with ISD versus Rag and what the rest of the industry uses. It's kind of by avoiding what is going on on Twitter and just keeping our heads down. So we do have a research lab. It is not a standard research lab. It does spend a lot of time on information retrieval and context.
00:38:02 · Matt Turck, George Sivulka
[00:38:02] S1: So do you think that, to take the extreme of what you said, that research is actually slowing down? So there's test time compute that we're talking about that sort of felt like the major innovation, you know, over the last six months. But after we're done with Test Time Compute, are we sort of running out of tricks? I think there there will [00:38:21] S0: be more tricks. I do believe that AI will start to work on itself. I think that... I don't mean to be relatively pessimistic, but I wouldn't start a company in AI right now. I mean that from the perspective of the alpha is gone. When I was starting the company, everyone was working on crypto. And it was like, you know, the alpha was gone. I was like, okay, we're gonna work on AI. And so I don't know what it is. It's like the next thing. But I do think that the models will continue to get better. They'll get way cheaper. They'll get way faster, and the user experience will improve. I think longer term, just as I'm talking about how generalization beats specialization, you saw over the last twenty years of enterprise SaaS, Excel get unbundled into a million specialization things. It only made Excel more powerful. But there will still be specialization as a second wave to create a lot of value. But I don't see lots of very large changes in the near future. But maybe I've also gotten spoiled from how much stuff has changed. So I'll have to think about that one. [00:39:25] S1: Great. So everyone, pivot to crypto right now. Announcing the rebranding of the event. Crypto driven