The MAD Podcast: How AI Gets Built — with Matt Turck
00:12:24 · George Sivulka, Matt Turck
The same will be true as you start to think through these agent employees as nodes and how you organize them, and whether or not you're... Where you are on the zero to 100% AI agent dialectic. [00:12:36] S1: So fast forward a few years, what does that all look like? Whether you call it 2030, let's say, not twenty years out, 2030. So we all, what, AI managers? We spend half hour days interacting with agents. Do we still interact with humans? What does that look like? [00:12:54] S0: I think that right now, we're already AI managers. The only difference is the AI takes a single step. So if you are an AI modern organization, you probably are using some sort of chat or rag application. And how good you are at prompting that AI is how good you are at managing that AI. And instead of of letting the AI go and prosecute a task over and over and over and self correct, it can only take a single step. As agents are rolled out, you'll actually start to see people that are really good at prompting, really good at defining a process, be the best managers, and actually be the best at extending whatever their agenda is in the organization, or making their function the best. And so I think that everyone will be prompting, and prompting is managing, and it will all blur pretty soon.
00:12:54 · George Sivulka, Matt Turck
[00:12:54] S0: I think that right now, we're already AI managers. The only difference is the AI takes a single step. So if you are an AI modern organization, you probably are using some sort of chat or rag application. And how good you are at prompting that AI is how good you are at managing that AI. And instead of of letting the AI go and prosecute a task over and over and over and self correct, it can only take a single step. As agents are rolled out, you'll actually start to see people that are really good at prompting, really good at defining a process, be the best managers, and actually be the best at extending whatever their agenda is in the organization, or making their function the best. And so I think that everyone will be prompting, and prompting is managing, and it will all blur pretty soon. [00:13:44] S1: And as a quick reality check, I think you mentioned somewhere that AI agents will contribute more to GDP than human workers within a decade. How backloaded is that? In other words, what's your sense as somebody who is in the proverbial trenches every day of the reality of agents in the enterprise, terms of what they can do and what's overhyped?
00:13:44 · Matt Turck, George Sivulka
[00:13:44] S1: And as a quick reality check, I think you mentioned somewhere that AI agents will contribute more to GDP than human workers within a decade. How backloaded is that? In other words, what's your sense as somebody who is in the proverbial trenches every day of the reality of agents in the enterprise, terms of what they can do and what's overhyped? [00:14:09] S0: We will still be deploying AI in its current form as a chatbot in ten years. I think the way to think about that is there's actually still plenty of businesses in New York that don't take credit cards. They only take cash. And organizational change and technological change will take a lot of time and a lot of effort. At the same time, most people use credit cards. And when things work, and they're not just experiments, they actually happen very fast. You're starting to see that with certain AI companies that are massively penetrating markets that have gone from zero to some double digit percentage of their SAM. And we're fortunate that Hebia is one of them. But at the end of the day, the stragglers, the long tail of adopters will still take time. And I think that to answer your question on the nose, everything is going to be backloaded, right? I think it'll happen in the decade. The change to credit cards happened over five to ten years, and now this change to just point and click, and credit cards happen even faster than that. But there will still be stragglers. It'll still take time.
00:14:09 · George Sivulka, Matt Turck
[00:14:09] S0: We will still be deploying AI in its current form as a chatbot in ten years. I think the way to think about that is there's actually still plenty of businesses in New York that don't take credit cards. They only take cash. And organizational change and technological change will take a lot of time and a lot of effort. At the same time, most people use credit cards. And when things work, and they're not just experiments, they actually happen very fast. You're starting to see that with certain AI companies that are massively penetrating markets that have gone from zero to some double digit percentage of their SAM. And we're fortunate that Hebia is one of them. But at the end of the day, the stragglers, the long tail of adopters will still take time. And I think that to answer your question on the nose, everything is going to be backloaded, right? I think it'll happen in the decade. The change to credit cards happened over five to ten years, and now this change to just point and click, and credit cards happen even faster than that. But there will still be stragglers. It'll still take time. [00:15:17] S1: One last question before we get into Fabian detail. What do you find particularly interesting in AI today, AI research, open source, reasoning models that, you know, you may or may not sort of import into Hebiah directly, but what do you find interesting in terms of like what's happening in research? [00:15:37] S0: You know,
00:15:37 · George Sivulka, Matt Turck
[00:15:37] S0: You know, [00:15:37] S1: there seems to be a new model every other day, a new thing every other day. What catches your attention? It's a good question.
00:15:43 · George Sivulka
[00:15:43] S0: There's the felt sense, I think, in communities like this one, and then maybe the larger AI community, that the scaling laws for training have slowed down a little bit. And we really haven't had a massive paradigm shift that has been released recently. At the same time, there's a lot of really interesting research direction in scaling laws for scaling during inference. And what that means is, hey, maybe we have a fundamental unit of compute, and that is a single inference, a single forward pass on a large language model. How do we actually now use that or run lots of inference like an agent, like OpenAI three, four, and all of these kind of more agentic, more reasoning models to actually get better at doing these difficult tasks. And actually the whole scaling at inference paradigm was pioneered at Hebia. So our early matrix product two years ago, we're one of the first people to say, Hey, you get way better accuracy from using more large language model calls at runtime. And so we built lots of infrastructure to scale that up. Actually built an agents team before was even called agents to actually go out and run these larger jobs. And I think that's probably the most interesting research direction moving forward is like, hey, let's go and say, this current scaling law of training has slowed down the scaling law of inference or test time compute is very interesting. Let's double click there. There's lots going on with multimodal research, obviously context windows there, how you apply them, how you tie them in to the larger ecosystem of foundation models that are used for text applications is very interesting. And I think the number one problem in all of AI is elongating the context window. And so we have a few different metrics. It's now a focus. You've got these needle in the haystack Twitter threads that measure how good they are. One of the areas that Hevia has actually been doing a lot of research has been in artificially extending the context window. We're effectively extending it. And that's been a massive, actually boon leveraging inference time compute for accuracy for research and diligence workflows.