Latent Space: The AI Engineer Podcast
01:24:41 · Shawn Wang, Jamin Ball
[01:24:41] S1: They could [01:24:41] S0: go have Dell package them up. They could raise the debt. And then you know what? There could be local community uprising and picketing. I'm literally... I'm not even being kidding. And then it's delayed. Then what? Well, the debt was raised. Someone's got to pay the interest. What happens? And you have creative delayed draw facilities where the debt is committed, but you don't draw it down. But all of these have limits, and you get super in the weeds on SLAs who was responsible for it. Was it a delay in the supply chain? Was it a delay in the construction, the data center developer? Was it a delay by the operator? Was it something else like community unrest? That's everything that credit markets are thinking through. And then there's a lot of other super technical reasons we don't need to get into that make it difficult with credit default swaps and hedges going through the roof. So now it's for debt providers, it's tough to hedge the exposure they have. And so as the hedges get more expensive, your willingness to take on more risk goes down because you can't hedge out of it. Like, it's [01:25:47] S1: This is all the function. This is all symptoms of the debt market running out of money. [01:25:51] S0: Yeah. Well, I don't I don't know if debt markets are super deep. I think it's more of like an appetite. And, of course, there's a price for everything. Hey. Great. You know what? Go take out 5,000,000,000 of debt at 20%.
01:25:51 · Jamin Ball, Shawn Wang
[01:25:51] S0: Yeah. Well, I don't I don't know if debt markets are super deep. I think it's more of like an appetite. And, of course, there's a price for everything. Hey. Great. You know what? Go take out 5,000,000,000 of debt at 20%. [01:26:00] S1: But you don't see this... You see this at equity layer in... Like, so, like, again, how come there's a $100,000,000,000 of equity Yeah. That's gonna go into Anthropic and and Anthropic cannot raise a $100,000,000,000 or $50,000,000,000? [01:26:11] S0: I I think... I would... I don't know if I would say they can't do it. Yeah. I don't know if... Well, one, they they need to be rated. Do they want to? Yeah. My guess is Anthropic and OpenAI are happy that NeoClouds exist to take the balance sheet risk. Maybe they don't want to take that balance sheet risk because, again, it could be things out of their control that blow them up. You have a huge debt stack in your cap table and things out of your control [01:26:39] S1: go Someone else's problem. [01:26:40] S0: Like, you're not the one holding the bag. And again, this is like the beauty and the curse of why Neo Clouds can be great businesses, or they could be brutal businesses if the contracts aren't airtight, if things aren't structured well, your balance sheet business, you know, and just like fintech balance sheet businesses can ebb and flow, know, there's a ton of... [01:27:02] S1: Oops. Computer may have died. It's fine. We'll edit. We'll edit this.
01:27:02 · Shawn Wang, Jamin Ball
[01:27:02] S1: Oops. Computer may have died. It's fine. We'll edit. We'll edit this. [01:27:05] S0: Okay. You know, balance sheet businesses, right? Like there were a ton of fintech companies in 2020 that did super well when interest rates were low, that got blown up when interest rates were high. And balance sheet businesses can be really tricky and very cyclical. And you don't want to own a balance sheet business when a market cycle tips over. So hey, on some level, I'm sure, you know, especially for internal chip efforts, right, like the OpenAI team will... Might... May wanna develop their own data centers for their jalapeno chips. Right? And Anthropic, same thing for their chips. But I'm sure they're also very happy for other people to take on that risk because it's someone else's core competency, and, you know, a lot of debt could add a lot of blow up risk. And there was no debt on on the Anthropic balance sheet. Now they have facilities in place that they could draw down. And so would it surprise me if you start to see debt on their balance sheet in the future? No. I mean... But, again, it's like, you wanna build a gigawatt. Right? $50.60, $70,000,000,000. That's that's not that's not a trivial amount just for... And that's just just one gigawatt. I think both OpenAI and Anthropic have said they kinda wanna get to five by the end of the year. Who knows if it's if it's that or something more or less than that? [01:28:18] S1: Are they about equal? [01:28:19] S0: I don't know. Okay. That's interesting to me.
01:28:19 · Jamin Ball, Shawn Wang
[01:28:19] S0: I don't know. Okay. That's interesting to me. [01:28:23] S1: Because I I think Anthropic OpenAI was ahead. Correct. Yeah. Right? By by a lot. Yeah. Exactly. But I think Anthropic slows the gap a lot. And OpenAI at at least started... Like, Stargate was a plan... The first stage in a plan to get to 30. Mhmm. 30 gigawatts. Mhmm. Mhmm. And so Like, twenty thirty. Yeah. So... So I was like, okay. Well, that's I I haven't heard that plan from Anthropic. [01:28:40] S0: But again, like, let's let's hit... Let's do some fun math really quick on the fly. Let's see if [01:28:44] S1: I can do it.
01:28:44 · Jamin Ball
[01:28:44] S0: Oh. Getting tired. OpenAI, let's say... I think when they came out with that, it was maybe a year ago, and they were what? Like, one and a half gigawatts, something like that. So whatever. Round that down to zero for just the sake of the argument. Let's say open and anthropic, each one go from zero to 30 over five years. So you're gonna add 60 gigawatts of AI capacity, just the two labs. What's a good assumption for their total capacity? I don't know, 40, 50%. Let's say it's 40%. And... Well, I don't know. Now we're getting too tough in math in my head, but let's say that gets you to, I don't know, 110, 120 gigawatts of total capacity brought online over the next five years. Let's say a 100, just so the math is easier. And let's say it costs $50,000,000,000 to build out a gigawatt. That's $5,000,000,000,000. $5,000,000,000,000 over the next five years. Right? And so call it a trillion a year. You're you're talking about a nontrivial percent of GDP, you know, real GDP. That's like 5% plus or minus of GDP. Like, that is a massive Bonkers. Massive build out. You know? Rivals, the real world build out. I think I've seen some charts somewhere that kind of compare those expected numbers, when you're measuring things in whole number percents of GDP, that's huge. And so like, the credit markets, debt markets need to be super, super, super deep and super, super open for that to happen. You need global build out. You know, it's kind of funny. I had a... You know, semi analysis number for next year is like 50, and I think Brad at the All In Summit said, oh, maybe it's lower, and I think there was some trolling going on back and forth. And I think the end result of that was like, hey, maybe the 50 gigawatt number, I don't want to speak for anyone, was more of like what's constructed. Brad was more of what's constructed and live generating tokens because there can be a gap in what's constructed and ready to go versus what's actually live and generating tokens. But I think it's like a real debate in the world. You know? And then you could see a pretty wide range of outcomes. You know? Do we really add 50 gigawatts of capacity next year as an industry? Or is it 20 or 30? Who knows? I just know everyone you talk to in the supply chain is saying, We have orders or indications for orders that are well over 100% of our capacity. Yeah.
01:31:05 · Jamin Ball, Shawn Wang
And, like, I don't have line of sight to that slowing down for the next few years. [01:31:10] S1: Yeah. And Yeah. And and the limit, like, LAN, Shell Power, all these, but also the silicon, like, straight up Totally. Like, compute memory, every every part of the supply chain. [01:31:19] S0: Totally. [01:31:20] S1: That's the work. That's why 78,000 is so powerful right now. [01:31:22] S0: I mean, look at these recent Kimi models. Kimi K three. You know what's memory? You need gigabytes of memory just to serve that. You're not running that on your laptop. You know, you need like a rack to serve one model. Yeah. Again, are just the big models. Right? Like you're also gonna have tons of small models, but I know the big models are getting bigger, which only require more memory and more compute to run. [01:31:44] S1: I used to say they were capping out. I used to say like the max size is one to two, and then, like, we would just optimize from there. Yeah. I've been wrong. [01:31:50] S0: What's what's interesting is it does feel like the open weight models are capping more around, a 100,000,000,000 active parameters. [01:31:55] S1: That's right. Yeah. And It's mostly, like, an inference speed thing. Totally. But, like, I [01:31:59] S0: think Totally.