Latent Space: The AI Engineer Podcast
00:17:34 · Shawn Wang, Jamin Ball
[00:17:34] S1: Let's let's do full screen on this one. Oh, yeah. Yeah. Yeah. Just just so... Yeah. It's Perfect. It's a little small. [00:17:39] S0: Yeah. Exactly. Alright. Here's a chart, and this is broken down in their s one, that shows the percent of their compute that goes towards basically serving models. So inference revenue generating or internal R and D kind of training or maybe safety and alignment or non serving of tokens. And what's interesting is you can see now that, I mean, they're at 27%. Only 27% of their compute today is going towards serving models. Here's why this is so interesting. If you compare the dollar spend in compute a year ago versus today, their compute that went towards serving models has gone up by 2.3 x. Their revenue over the same period of time has gone up 14 x. [00:18:22] S1: So [00:18:23] S0: embedded in that is basically a seven x efficiency gain in serving inference, which... I mean, this is a business that I think is almost accidentally profitable. Right? Where... I mean, a seven x increase in efficiency of serving tokens, If you were monetizing a megawatt at $1,015,000,000 dollars, now you're monetizing it at 70 to $100,000,000 And here's... This is a little bit of a random rant. But software companies have historically told the world and told investors, hey. We're an annuity of cash flows. Once we acquire a customer, our net retention's high and they'll they'll stick [00:19:00] S1: with Cost of a lifetime value.
00:19:00 · Shawn Wang, Jamin Ball
[00:19:00] S1: with Cost of a lifetime value. [00:19:01] S0: Exactly. We're gonna be burning a lot of money early on, but we're just in growth mode. We're in customer acquisition mode. Once we exit growth mode, we'll be more in harvest mode. And that's when we'll start to become... We'll flip profitable because we're not spending a lot of money on customer acquisition costs. But, you know, the general model of a SaaS company was unprofitable over time, then profitable. What's happened with Anthropic, and let me pull up another chart here, I believe I have their... Here are their free cash flow margins. Oh my God. [00:19:33] S1: It's so negative. [00:19:34] S0: Which I had to cut the chart. Right? Like negative 4000%, negative 1500%, you know, super, super negative. If if I actually drew the axes to scale, it it would... It wouldn't look real. And so, yes, the the axis is broken. But this is the free cash flow margin of Anthropic. 21 free cash flow margin in the most recent quarter. And so what you're seeing is a business that all of a sudden, because they were able to drive such efficiency in their compute, that's [00:20:02] S1: kind of why I went
00:20:02 · Shawn Wang, Jamin Ball
[00:20:02] S1: kind of why I went [00:20:03] S0: on that compute ramble, they've been able to monetize at much higher rates and actually deliver positive free cash flow margins pretty early on. And so this is a business that says, hey, we've proven that we can operate highly profitably. What we've probably done is underinvested in compute and compute build out, and we need to be more aggressive on it. And so my guess is that you're actually going to see this level off and maybe go negative again as they say, Hey, we're going to reinvest, and we're going to invest in a lot of compute. We're going to build out a lot of compute. Because now they've shown themselves and the world [00:20:34] S1: The capability. [00:20:34] S0: We need to, we can can flip the switch and and be profitable.
00:20:34 · Jamin Ball, Shawn Wang
[00:20:34] S0: We need to, we can can flip the switch and and be profitable. [00:20:39] S1: Yeah. And for listeners, a couple things on the accounting side, just because I... We gotta educate people. Right? FCF is typically, like, the most honest version of what profit looks like. Yeah. But in this case, you can mostly just regard it as what an operating margin would... Profit margin would look like because, basically, FCF just focuses on cash, and so there's less tricks to play with accrued receivables and and and offsetting your liabilities and what have you. And so, like, that that is the roughly Yeah. Rough thing. And... But also this... The re... The reason people have adjusted revenue and adjusted profit is because of nuances in the running the business like this. Right? Like, GAP, which is the official generally accounted... Accepted accounting principles that you have to report, doesn't doesn't regard how you want to run the business and split out investment. It doesn't have investment decisions versus, like, sort of profitability of of core business decisions, but, it all gets smushed into these decisions. Right? It's like... So, basically, what what I'm trying to sort of guide people towards, if you do see a change in this margin profile, it is a conscious choice. Yeah. And, actually, you should probably split it out further between what is actually the core business and, you know, is that actually inflecting Mhmm. Or is this more investment, therefore, the FCF went down?
00:21:58 · Jamin Ball, Shawn Wang
[00:21:58] S0: Yeah. And look, at the end of the day, cash is king. And your measure of profitability should be, is your cash balance going up or is your cash balance going down? Now, if you wanted to poke holes at this, reality of GAAP accounting is there's two forms of... Well, very broadly, there's two forms of expense. There's the cash and the salary that you pay people, and then there's also the [00:22:18] S1: equity. Stock based. [00:22:19] S0: And then the non cash that you pay people. So non cash stock based compensation, equity compensation, that doesn't show up in free cash flow. It's actually added back because it's a non cash expense. So there are a lot of people who look at the free cash flow metric and say, well, it actually overstates your profitability because the reality is if you didn't grant equity to your hires, they wouldn't join you. And so it is a real expense. Treat it as a real expense. I don't have a chart for this, but if you were to adjust free cash flow [00:22:47] S1: It's still profitable. [00:22:47] S0: Stock based comp, you're still at an 8% Yeah. Free cash flow excluding stock based comp. And so you're you're even profitable if you include the big By normal [00:22:56] S1: rules. Yes. [00:22:56] S0: Yeah. Exactly. [00:22:57] S1: And so, like, if you've never seen a software IPO, this never happens. [00:23:00] S0: Yeah. It's very rare. I mean, I think there are there are a lot of software companies that are free cash flow positive today. [00:23:06] S1: Yeah. [00:23:06] S0: But if you factor Not at IPO. Never. Not at IPO.
00:23:06 · Jamin Ball, Shawn Wang
[00:23:06] S0: But if you factor Not at IPO. Never. Not at IPO. [00:23:09] S1: Never at IPO. [00:23:10] S0: But but even today, the mature ones, even if you factor in stock based comp, most of them are free cash flow neutral or negative if you treat stock based comp as a cash expense. [00:23:19] S1: So all those companies are dying and everyone's just buying Anthropic, right, as the company? [00:23:23] S0: I think there's a... Well, you're not quite that, but certainly [00:23:27] S1: Why would you? Like, anything else.