The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
00:08:30 · Parag Agrawal
[00:08:30] S0: So you allow the agent to specify that in the API signature. So our product is an API, which either the programmer can configure based on their application or can leave it to the agent. Our API even has a parameter, which is like, what's the model calling me? If you know the model, we can do things differently. Now you don't have to tell us, but if you tell us, you might get better results. We ship just like you can use, like, a model. You can use a small model or a big model, and you can use it with, like, low thinking or medium thinking or high thinking. We have productized our search system into a few different modes, each optimized for a certain class of use case. For example, we have a really, really fast API, the fastest in the market. It's fast and cheap because with low latency budget, there's only so much compute you can do. And it's built for voice agents. So your voice agent must be all knowing without telling you, guys, I'm searching the web. Hold on while I come back with the answer. Like, that's a silly experience for a voice agent. Right? But you should just, like, magically immediately respond. And you now need to do web search, which is like a rapid. On the other hand, you can take a Fable model, and the thing is gonna think for twenty seconds and then generate for it. You can spend five seconds on web search to make sure it does into five web searches, only two. So you end to end in the agent, you save time and cost. And so that's a different processor on our system. It's called advanced. So you use Turbo if you're building a voice agent, you use advanced if you're like really expensive background agent.
00:09:59 · Harry Stebbings, Parag Agrawal
[00:09:59] S1: And so the primary use case today in terms of customer base is engineering and coding. [00:10:04] S0: It's pretty broad based. I would say the primary use cases, the common theme is knowledge work. Coding is a category of knowledge work. So are AI lawyers. So are productivity applications. So are AI insurance underwriters. So are [00:10:18] S1: AI scientists. I'm trying to understand how much of the mother load is engineering. Is it 80%? [00:10:24] S0: No. The the thing about engineering is coding is a large chunk of inference in the market right now. Coding invokes, I would say, web search in 5% of prompts. Woah. Right? So it's not every prompt invoking web search when you're writing code because most of them rely on your internal context and your internal data and your code base. So the model is spending time reading your internal code and not on the web. Law is not gonna [00:10:52] S1: be much more, is it? [00:10:53] S0: Law is very web search oriented. [00:10:55] S1: Really? I thought it'd be internal data driven. [00:10:57] S0: There is internal data, but there is case law. There is facts. There's facts about companies, facts about people, and you have to go exclude information. So you have to be very comprehensive in law to say, I want to be confident that despite a lot of effort, you can't find this. Insurance underwriting has the same flavor. So there's... Sales is very web search heavy. AI science is very web search heavy. So on a relative basis, inference goes more in compute. But when you think of web search, all of these others start popping too.
00:10:57 · Parag Agrawal, Harry Stebbings
[00:10:57] S0: There is internal data, but there is case law. There is facts. There's facts about companies, facts about people, and you have to go exclude information. So you have to be very comprehensive in law to say, I want to be confident that despite a lot of effort, you can't find this. Insurance underwriting has the same flavor. So there's... Sales is very web search heavy. AI science is very web search heavy. So on a relative basis, inference goes more in compute. But when you think of web search, all of these others start popping too. [00:11:26] S1: We were talking before about muse and instinct. If there's anything that needs web search, it's personal assistance. It's great. How does that... Yeah. It's great. This is like, woo hoo. This is the greatest thing for my business. How does that change your business?
00:11:26 · Harry Stebbings, Parag Agrawal
[00:11:26] S1: We were talking before about muse and instinct. If there's anything that needs web search, it's personal assistance. It's great. How does that... Yeah. It's great. This is like, woo hoo. This is the greatest thing for my business. How does that change your business? [00:11:40] S0: In our business, right, anytime agents start becoming more useful for more use cases because models either get better or cheaper, it's great for us. Right? Because we bet that agents will be the consumers for the web, and we're we've been building tech for agents. So when agent do more, it's great for our business. So we want models to keep getting better and cheaper so that agents do more and more and more. And if that happens, it's great for our business. If models get smarter, don't the agents beneath them do fewer searches, and then it's worse for your business? I don't think so. If you think of models, there is the trend around models being smarter and then models having more memorized than parametric memory. Like, those are two slightly different dimensions. Today, by and large, I would say models have good recall from parametric memory on, let's call them head facts. Facts. It's like for somebody famous, everything about them, Wikipedia, it can memorize. So it'll tell you who the president was in a certain year. Right? Because a model can memorize those things. The model can tell you what year I graduated from college. Maybe for me it can, but it can't tell you for somebody who works at parallel. [00:12:56] S1: Do you know things if I if I... I mean, if I were
00:12:56 · Harry Stebbings, Parag Agrawal
[00:12:56] S1: Do you know things if I if I... I mean, if I were [00:12:58] S0: to Even if it was in the pretrained data. Seriously? Because it's lossy compression. So what a model's parametric memory is doing? It's lossily compressing to understand patterns in the world. And so it can't memorize every fact in pretraining data. So one, not every fact is in pretraining data. Two, it can't... For even stuff that's in pretraining data, the model's actually trying to find patterns rather than memorize them. And then further, as you make models efficient, which is you make them smaller and smaller while keeping the performance by distilling them or whatever, you lose more of the parametric memory where you try to keep the reasoning. [00:13:33] S1: Do you think we will see models become smaller and smaller than every company have their own model with their own data and the fireworks theory of, you know, own your own intelligence being true? [00:13:44] S0: There are two questions in there. So one, think we're gonna see two things. We're gonna see the biggest or the frontier models be bigger and bigger over time. We are also going to see smaller and smaller models being able to reach any fixed level of performance. So if you say, okay, I want opus 4.8 level of performance. Okay? That's good enough for my use case. Every six months, a much smaller model will be able to deliver that to you. The useful range of sizes of models will be way, way, way different. What does it [00:14:14] S1: mean if the frontier get bigger and bigger? What are the ramifications of that?
00:14:14 · Harry Stebbings, Parag Agrawal
[00:14:14] S1: mean if the frontier get bigger and bigger? What are the ramifications of that? [00:14:18] S0: They are better. All of the ramifications you imagine. The reason the frontier will get bigger and bigger is ultimately the gap between the value for certain use cases incremental quality can provide you. Can be so high in certain use cases, which can be so valuable that it's worth paying for. So if you can build it, there will be use cases for it as long as by making it bigger, you can make it better. It appears there is no end to the scaling law that we can perceive so far. Again, I'm conflating bigger with like models are getting bigger, but they're also able to think longer. So you're just able to throw more compute at the same problem. That will keep happening. I think we'll be able to throw more and more compute at same problem and make the answer be marginally better over time. And so we're gonna just spend a lot of money on extremely large models solving really hard problems. [00:15:11] S1: Do you agree with the consensus view that you'll have 90% of token activity go through open models, but 90% of dollars go through frontier models?