Latent Space: The AI Engineer Podcast
00:28:55 · Shawn Wang, Jamin Ball
[00:28:55] S1: You know, like, have you read the rest of the s one nine? Well, what I [00:28:57] S0: what I... So what I tried to do what I tried to do in this chart is I broke down the the... Because they they broke down their commitments. They didn't do it by year, but Is [00:29:07] S1: this normal, by the way, that you would have year by year? [00:29:09] S0: No. Feel like this is unnecessary. Year year by year, this is a little bit of the Jamin best interpretation. They'll give a dollar. I've seen variations of [00:29:18] S1: this. Yeah. Yeah. Yeah. [00:29:19] S0: It's... And so I think what you have to ask yourself is in the commitments that they've made, and I've tried to annualize the commitments. If they said this is a five year deal, I've just made it, This is a five [00:29:28] S1: year deal. By the way, every banker in the world has done this already. Yeah. Like, they did this on the night of the S1. Right? [00:29:34] S0: So you have to ask yourselves, can you pay for it? Okay. Well, at a fixed basis, and then the gray bars is maybe like less hard commits, but commits they wanna make. There's just [00:29:45] S1: the x a... SpaceX one is in there. Right? Yeah. Exactly. They can walk away from nineteen
00:29:45 · Shawn Wang, Jamin Ball
[00:29:45] S1: the x a... SpaceX one is in there. Right? Yeah. Exactly. They can walk away from nineteen [00:29:49] S0: ninety days. Yeah. Will they be able to fund a 100,000,000,000 a year? And So it isn't $850,000,000,000 a year, it's can they fund $100,000,000,000 a year? Now, there's a couple ways they could do that. Well, one, do you expect their growth to continue? Which is a question of will there continue to be a business at the frontier? Or you go back to the, well, only 25% of their compute today is allocated towards revenue generating or kind of inference serving use cases. If they were to just double that and add zero compute, they would... They could, in theory, double their revenue. And so, like, you look at the headline number of 855,000,000,000, you're like, oh my gosh. That's a huge number. How are gonna do it? Some of that is very back weighted. But I I I think when I think about the trajectory of the business, the TAM that's been penetrated today, I don't think it's far fetched at all to say, hey. Like, the business trajectory will more than cover... The revenue will more than cover the the expense. I think I have another chart in here. This is another great one. This is the compute margin. So this looks at if you were to take their revenue and subtract all compute, internal models training [00:31:06] S1: Same quarter matched.
00:31:06 · Shawn Wang, Jamin Ball
[00:31:06] S1: Same quarter matched. [00:31:07] S0: Exactly. So even as... What you're seeing is even as they've shifted more of their compute to non revenue generating use cases, they've still scaled their compute margin to well over 50%. What you're... So what you're seeing is they're getting hyper efficient at the compute they bring online. The compute they decide to use for revenue generating use cases is becoming... It's become seven x more efficient over the last year. I think you have to ask yourself, will they continue to ramp compute and capacity? I think the answer is yes. I think if you ask anyone at Anthropic, again, this is all speculation. They probably say, we wish we started doing it sooner. We wish we were more [00:31:46] S1: They're trying to commit to more compute. That's that's [00:31:48] S0: that's right. [00:31:49] S1: If you don't talk to any of these guys, you come away with, like, oh my god. They're... You know, they... These guys are absolutely crazy and being fiscally irresponsible. Yeah. But I think if you have any confidence or if you see any... If you see up close the the demand for intelligence that people are snapping up and immediately consuming whatever you give them Mhmm. Then, like, yes, actually, you're trying to get more compute. Yeah. That... That's... The the game is not to reduce the commitments. It is actually to get more commitments. Yeah. They they would have a trillion if they could.
00:31:49 · Shawn Wang, Jamin Ball
[00:31:49] S1: If you don't talk to any of these guys, you come away with, like, oh my god. They're... You know, they... These guys are absolutely crazy and being fiscally irresponsible. Yeah. But I think if you have any confidence or if you see any... If you see up close the the demand for intelligence that people are snapping up and immediately consuming whatever you give them Mhmm. Then, like, yes, actually, you're trying to get more compute. Yeah. That... That's... The the game is not to reduce the commitments. It is actually to get more commitments. Yeah. They they would have a trillion if they could. [00:32:17] S0: Yeah. I don't think I've spoken to anyone in the AI industry, certainly not anyone with the big labs who has said, yeah, we're, you know, I feel [00:32:23] S1: We're good. We're the compute we have. [00:32:25] S0: It's almost always... And and I think a underrated factor on compute demands in the future will be compute that's used for safety, alignment, and verification. Like, I think more and more compute is going to go towards those functions, whether that's on research Yeah. Or just on verification and alignment that happens at inference time, that will be an increased compute demand for all of [00:32:50] S1: these things. Probably not gonna break those out, you know, but, like, it'll it'll be, like, maybe 5% now and [00:32:55] S0: Yeah. Of course. [00:32:55] S1: 20% in future. [00:32:56] S0: Of course.
00:32:56 · Jamin Ball, Shawn Wang
[00:32:56] S0: Of course. [00:32:58] S1: I think, you know, I I would I would just tell people, like, if you think this is bad now, wait till you see the OpenAI as one. Yeah. And then, like, it's not bad. It's... Like, that's the whole point. It's like they're actually really trying to get compute from every source they can get Mhmm. And, like, OpenAI is buying TPUs. Yeah. Yeah. Like, there was a cold day in hell that never happened. [00:33:16] S0: TPUs. Right? They have their own chip initiative, Jalapeno, which has pretty rave reviews. Uh-huh. Anthropic staffed up a compute team to build their own chips. Obviously, the hyperscalers are building their own chips. It's I mean A [00:33:28] S1: mad dash. [00:33:29] S0: It's a it's a mad dash. [00:33:30] S1: Only, you know, only winners are TSMC, NVIDIA, and ASML Yeah. Whatever, in the supply chain stack. [00:33:37] S0: Totally. And then I think, look, it's it's getting harder and harder, especially in The United States. You have a lot of pushback. I mean, you have all kinds of pushback. Data centers are too noisy. Don't want them in my neighborhood. They add a lot of air pollution. The water. Don't want it. It's gonna... The water. [00:33:49] S1: Oh my god. The
00:33:50 · Jamin Ball
[00:33:50] S0: water. The water. It's... Which is... It uses too much water, which it doesn't. It's the electricity. It uses a lot of electricity. It's gonna increase my electricity bill if I live in close proximity. Or it's just the simple AI is going to kill us and I don't want to enable that, which again, you have all kinds of pushback. It's become hyper hyper political, but I think the unfortunate thing is it has become a lot harder. You do see a lot more pushback. Midterms are coming up. Right? Who who kind of like takes control in, like, the house and the senate? Like, could have a big impact on regulation or lack thereof around data center and intelligence center build outs. And the good thing is that it's getting harder and harder to build these data centers. Behind the meter is harder and harder. And so you do have a natural governor on the build out, and you're... We're... I think we're a lot less likely to overbuild too quickly that leads to a supply glut. Right? Kind of the the Internet bubble analogy of the early two thousands where we built way too much fiber. There wasn't as much demand for the Internet. There was a lot of dark fiber. Right? Fiber that had no bits and bytes traveling over it. We have no dark GPUs today. So And it's it's probably a good thing that there's some rate limiter. It will stop us from entering a huge bubble, but it's also bad because we need... You know, the industry needs more compute, especially as these models get more and more powerful. You know, the security concerns go through the roof. You also need a lot more... Like, the defenders need more compute. The defenders need more capabilities