Latent Space: The AI Engineer Podcast
00:24:57 · Shawn Wang, Jamin Ball
[00:24:57] S1: Yeah. And, you know, I think the the word is virtual collaborators. Yeah. You know, I I say agents, but Right. You you make it a friendly term for people. And, you know, I really start to tell people, you should just start thinking about it in terms of, like, what your employee spend versus your workers spend is. [00:25:12] S0: Mhmm. [00:25:12] S1: Mhmm. And most people are at, let's call it, 0.1 or Yeah. Zero point... Or, like, one to 5%. Yep. The median AI company is at 50%. Mhmm. And the top AI companies... You know, let's say ninetieth percent of telanthropic person is, 10 x token Yeah. Tokens of salaries. Yep. And so that is probably the right comp of, like, well, what are you spending on humans? Yeah. Right. And like, probably 30%. Right? Yeah. Like Yeah. And well... Okay. [00:25:38] S0: So Could be more even. I mean, yeah, humans could be like four... You know, total headcount could even [00:25:42] S1: be like 40% the day. Right? Yeah. Exactly. And so, like, if you're a physical business, fine. That's like, until we have good robots, then that's not that's not getting happening. But what are you spending on knowledge worker humans? What is... And and knowledge worker humans that includes coders, but also just admin and ops and what have you and and and legal and whatever you want Yeah. And all of that. And some part... Like, that percentage... The the ratio of tokens spend to human spend was zero to whatever Yep. And now it's gonna rise. Yeah. And how far can it go?
00:25:42 · Shawn Wang, Jamin Ball
[00:25:42] S1: be like 40% the day. Right? Yeah. Exactly. And so, like, if you're a physical business, fine. That's like, until we have good robots, then that's not that's not getting happening. But what are you spending on knowledge worker humans? What is... And and knowledge worker humans that includes coders, but also just admin and ops and what have you and and and legal and whatever you want Yeah. And all of that. And some part... Like, that percentage... The the ratio of tokens spend to human spend was zero to whatever Yep. And now it's gonna rise. Yeah. And how far can it go? [00:26:10] S0: Yeah. Totally. Totally. It can go a lot. I mean, look. I I think there's there's a lot of, like, hyperbolic statements being made both positive and negative about OpenAI, Anthropic, the big labs. I think it really boils down to three main questions. Can Anthropic stay on the frontier? Is there a business and a durable business to be built at the frontier? And if so, how much is it gonna cost? And and and and then in reality, two and three kind of, like, relate to each other. Can it make enough money to cover all the costs to be at the frontier? And that's that's the one where... I think I actually have a couple. I think I [00:26:48] S1: have one chart on this.
00:26:49 · Jamin Ball
[00:26:49] S0: Let me let me see which one. What did I want to bring up? This one... Yeah. Let me pull up... No. Let me pull up this one. This is a good one. So this is kind of the second question. Like, how much will it cost to be at the frontier? Or or or even maybe let's go to, like, what what can you build a business at the frontier? The other interesting stat that Anthropic had in their s one was for the three months ending September, 80% of their revenue was driven by models that were six months old or younger. And so what does that tell you? The vast majority of their revenue is coming from their most recent, most powerful models. And that's something that I kind of expect to continue. I think, you know, when it comes to frontier tokens down to open weight tokens, I think there'll be a massive market for both. I think the trend will continue to be on a... As a percent of tokens, open weight can... Continues to eat share. But as a percent of wallet share in dollars, Frontier continues to take the lion's share. So I expect Frontier tokens to be the most economically valuable to capture the most market share. I expect open weight and cheap tokens to capture the most market share in terms of total tokens. And I think kind of this stat on 80%, which is much bigger than I would have expected just in models six month old or less, kind of demonstrates the value of being on the frontier. Okay. So then let's go... Next, go to the second question, which is, well, what is it gonna... Or the third... Like, is it gonna And here's a chart, I think... And there... There's... I'll I'll have to caveat this chart. But a lot of people look at the Anthropic S1 and say, oh my gosh. There's $856,000,000,000 of Off the books. Of, like, not even off, but, like, committed deals that they've signed. Yes. How are they gonna pay for it? You know, they're at whatever whatever their ARR is, something in, like, I don't know, 70, 75, 80, you know, whatever. I think the last number was 65. The q three number was 16 0.8, but whatever. 70 plus or minus. How are you gonna pay for 855,000,000,000?
00:28:55 · Shawn Wang, Jamin Ball
[00:28:55] S1: You know, like, have you read the rest of the s one nine? Well, what I [00:28:57] S0: what I... So what I tried to do what I tried to do in this chart is I broke down the the... Because they they broke down their commitments. They didn't do it by year, but Is [00:29:07] S1: this normal, by the way, that you would have year by year? [00:29:09] S0: No. Feel like this is unnecessary. Year year by year, this is a little bit of the Jamin best interpretation. They'll give a dollar. I've seen variations of [00:29:18] S1: this. Yeah. Yeah. Yeah. [00:29:19] S0: It's... And so I think what you have to ask yourself is in the commitments that they've made, and I've tried to annualize the commitments. If they said this is a five year deal, I've just made it, This is a five [00:29:28] S1: year deal. By the way, every banker in the world has done this already. Yeah. Like, they did this on the night of the S1. Right? [00:29:34] S0: So you have to ask yourselves, can you pay for it? Okay. Well, at a fixed basis, and then the gray bars is maybe like less hard commits, but commits they wanna make. There's just [00:29:45] S1: the x a... SpaceX one is in there. Right? Yeah. Exactly. They can walk away from nineteen
00:29:45 · Shawn Wang, Jamin Ball
[00:29:45] S1: the x a... SpaceX one is in there. Right? Yeah. Exactly. They can walk away from nineteen [00:29:49] S0: ninety days. Yeah. Will they be able to fund a 100,000,000,000 a year? And So it isn't $850,000,000,000 a year, it's can they fund $100,000,000,000 a year? Now, there's a couple ways they could do that. Well, one, do you expect their growth to continue? Which is a question of will there continue to be a business at the frontier? Or you go back to the, well, only 25% of their compute today is allocated towards revenue generating or kind of inference serving use cases. If they were to just double that and add zero compute, they would... They could, in theory, double their revenue. And so, like, you look at the headline number of 855,000,000,000, you're like, oh my gosh. That's a huge number. How are gonna do it? Some of that is very back weighted. But I I I think when I think about the trajectory of the business, the TAM that's been penetrated today, I don't think it's far fetched at all to say, hey. Like, the business trajectory will more than cover... The revenue will more than cover the the expense. I think I have another chart in here. This is another great one. This is the compute margin. So this looks at if you were to take their revenue and subtract all compute, internal models training [00:31:06] S1: Same quarter matched.
00:31:06 · Shawn Wang, Jamin Ball
[00:31:06] S1: Same quarter matched. [00:31:07] S0: Exactly. So even as... What you're seeing is even as they've shifted more of their compute to non revenue generating use cases, they've still scaled their compute margin to well over 50%. What you're... So what you're seeing is they're getting hyper efficient at the compute they bring online. The compute they decide to use for revenue generating use cases is becoming... It's become seven x more efficient over the last year. I think you have to ask yourself, will they continue to ramp compute and capacity? I think the answer is yes. I think if you ask anyone at Anthropic, again, this is all speculation. They probably say, we wish we started doing it sooner. We wish we were more [00:31:46] S1: They're trying to commit to more compute. That's that's [00:31:48] S0: that's right. [00:31:49] S1: If you don't talk to any of these guys, you come away with, like, oh my god. They're... You know, they... These guys are absolutely crazy and being fiscally irresponsible. Yeah. But I think if you have any confidence or if you see any... If you see up close the the demand for intelligence that people are snapping up and immediately consuming whatever you give them Mhmm. Then, like, yes, actually, you're trying to get more compute. Yeah. That... That's... The the game is not to reduce the commitments. It is actually to get more commitments. Yeah. They they would have a trillion if they could.