Latent Space: The AI Engineer Podcast
00:12:48 · Jamin Ball, Shawn Wang
[00:12:48] S0: That's right. [00:12:49] S1: That's right. And so, like, that that is, like, a really interesting governing mechanism of democracy versus capitalism Yeah. Where, like, capitalism encourages centralization and democracy encourages decentralization somewhat. Anyway, so there... There's all that. I think it's very, very valid. I'll point out for people who don't know about capital structure that equity is is junior to debt. Yeah. And so it's nice to have 2,000,000,000,000 in [00:13:10] S0: in junior Yeah. Exactly. [00:13:13] S1: In your enterprise value. [00:13:14] S0: Right? Exactly. [00:13:14] S1: Exactly. That's like, well, that... That's that's that's public. And I think part of that also, I'll I'll throw it in the way now, which is a little bit more advanced than than people know, is interest rates are spiking Mhmm. For for for data center debts globally. Right? Like, is there a source to look that kind of stuff up? Is it just Bloomberg? Or is Yeah. [00:13:36] S0: I I mean, mean, there's [00:13:37] S1: just charts that people tweet around. But [00:13:39] S0: Most of these big debt deals will price, and they'll be, you know, like an interest rate, an OID. You know, there'll be a bunch of things associated, like, with the price, and, like, oftentimes, those will be public once they've priced. But, yeah, rates are going up. Risk free rates are going up. The Fed paused. You're... Decided not to hike yesterday or earlier this week. I don't know if that's necessarily a reason to go public. But
00:13:39 · Jamin Ball, Shawn Wang
[00:13:39] S0: Most of these big debt deals will price, and they'll be, you know, like an interest rate, an OID. You know, there'll be a bunch of things associated, like, with the price, and, like, oftentimes, those will be public once they've priced. But, yeah, rates are going up. Risk free rates are going up. The Fed paused. You're... Decided not to hike yesterday or earlier this week. I don't know if that's necessarily a reason to go public. But [00:14:07] S1: It's part of the why now. If if if it does help with that, that if my debt went from, like, I don't know, 3% to five percent, that is billions of dollars. Yeah. Yeah. And if I can keep a lid on that, that's very important to me. [00:14:21] S0: Yeah. I mean, a lot a lot of the data center debt deals now are are getting done over 10%. Yep. And so it's it's getting it's getting expensive. [00:14:28] S1: Okay. It's getting expensive. So... And then the other... That's the other part of this discussion, which is the ROI on a data center. Mhmm. Right? Because that... There there are a lot of AI skeptics Yeah. Who are like, this is the biggest build out in human history. Yeah. And we are throwing money after this illusory dream of AI. [00:14:47] S0: Yes. [00:14:47] S1: Then on the other side, the AI pro people are saying, well, your payback period is one year. Yeah. Yeah. Why is there such a huge difference? This
00:14:47 · Shawn Wang, Jamin Ball
[00:14:47] S1: Then on the other side, the AI pro people are saying, well, your payback period is one year. Yeah. Yeah. Why is there such a huge difference? This [00:14:56] S0: was kind of the best thing about the Anthropic S1, which is a lot of the metrics that were leaking where your ARR, you're this or you're that, but you didn't really get to see a lot of below the line. [00:15:04] S1: That's not ROI. [00:15:05] S0: Yeah. You didn't get to see the cash. How much are they burning? It's not profit. Are they profitable? Are they not? And that I think was one of the biggest surprises of the S1. Here's a business that showed 85% gross margins, over 20% free cash flow margins, 8% GAAP operating margins. And so even on a Yes. GAAP basis highly profitable. And maybe my my favorite stat of the whole s one was a $141,000,000,000 of capital raised to date. That's inclusive of equity, converts, and safes, and a $134,000,000,000 Cash in balance today. [00:15:42] S1: And so Cash and cash equivalents. But...
00:15:42 · Shawn Wang, Jamin Ball
[00:15:42] S1: And so Cash and cash equivalents. But... [00:15:44] S0: Yes. Exactly. And so you think, okay. Well, you've you've raised this much. You have this much in the bank. That means you've burned $7,000,000,000 to get to, you know, the q three numbers that aren't in the s one, but that were kind of like broadly reported a few weeks ago. $16,800,000,000 of quarterly revenue off of $7,000,000,000 of lifetime burn, which... And the burn we should we should get into because like that was a a huge debate on like, well, it looks like they did raise debt or are they really profitable? What is this net loss on the P and L? There's... I'd say lots of [00:16:17] S1: Yeah. [00:16:18] S0: So lots of things out in the world that aren't fully true. [00:16:20] S1: That is important. I do wanna cover it. Let's... That that... There's a little bit more in the weeds than than than at the start of this conversation. Yes. But, like, that overall efficiency is by far, if you've been following the relative fates of... Let's call it SpaceX Anthropic OpenAI Have you heard? The the ones that are going public, they are the most efficient lab in the world Yeah. By by, like, order of magnitude. [00:16:43] S0: Yeah. Totally.
00:16:43 · Jamin Ball, Shawn Wang
[00:16:43] S0: Yeah. Totally. [00:16:45] S1: And, you know, so I I... When... And when I wrote up the s one, I was like, okay. Well, this is the the the the mystery. Yeah. Like, why? What... What's going on? Like, how are they so much more efficient? Yeah. And I... They they... In their s one, they attributed it to the loops. Yep. The improvement loop, the investment loop, something like that. Yeah. Exactly. So people can go read that up. I I already wrote it up. But, like, you know, maybe they're just smarter. [00:17:08] S0: Yeah. Well, I I think here... Like, what what are the best numbers to look at is, like, what is their growth relative to their... What is the change in growth relative to the change in serving compute spend? And they break out their compute spend into more training and internal use [00:17:25] S1: And by way, versus you have charts to [00:17:27] S0: Yeah. Let's... I do have a chart on that. So here is here is a chart that shows the [00:17:34] S1: Let's let's do full screen on this one. Oh, yeah. Yeah. Yeah. Just just so... Yeah. It's Perfect. It's a little small.
00:17:34 · Shawn Wang, Jamin Ball
[00:17:34] S1: Let's let's do full screen on this one. Oh, yeah. Yeah. Yeah. Just just so... Yeah. It's Perfect. It's a little small. [00:17:39] S0: Yeah. Exactly. Alright. Here's a chart, and this is broken down in their s one, that shows the percent of their compute that goes towards basically serving models. So inference revenue generating or internal R and D kind of training or maybe safety and alignment or non serving of tokens. And what's interesting is you can see now that, I mean, they're at 27%. Only 27% of their compute today is going towards serving models. Here's why this is so interesting. If you compare the dollar spend in compute a year ago versus today, their compute that went towards serving models has gone up by 2.3 x. Their revenue over the same period of time has gone up 14 x. [00:18:22] S1: So [00:18:23] S0: embedded in that is basically a seven x efficiency gain in serving inference, which... I mean, this is a business that I think is almost accidentally profitable. Right? Where... I mean, a seven x increase in efficiency of serving tokens, If you were monetizing a megawatt at $1,015,000,000 dollars, now you're monetizing it at 70 to $100,000,000 And here's... This is a little bit of a random rant. But software companies have historically told the world and told investors, hey. We're an annuity of cash flows. Once we acquire a customer, our net retention's high and they'll they'll stick [00:19:00] S1: with Cost of a lifetime value.