Invest Like the Best with Patrick O'Shaughnessy
00:27:40 · Gavin Baker
[00:27:40] S2: does. It's just wild. But yeah, I mean, I think there's a long list of pretty obvious things. If the sum of these labs plateaus or starts to decline, that's really negative, unless it's just because open source tokens are net growing the pie and taking share. And I do really think the future is multimodal, particularly for the AI natives, they're going to want to take an open source model, it's got all these inference clouds have gotten really good at supervised fine tuning and reinforcement learning, so you can take your data, custom- As an open source model, and then get something that you can put behind a router, and the router routes it to, often first, your model, and then Claude, Frontier model, whatever Claude Grok, checks it, and you can, in a lot of cases, get slightly better outcomes At half the cost. But again, that half the cost, I think a lot of people hear that, they're like, "That's bad for AI demand." It's actually not at all, because the cost the user pays is just a function of the margin on the tokens, and you're literally just shifting tokens from really expensive tokens with like ninety percent gross margins to tokens with maybe, let's call it a thirty percent gross margin. That's where the savings are coming from, but the tokens cost the same amount of compute To produce. And then also all these things are kind of happening on different cycle times. All these big public companies are like, "Oh my God, my AI spend is twenty X, I've burned my budget in three months," so they set up a router, and that actually cuts their AI spend, but it doesn't really impact, it may actually increase the amount of tokens that they are generating just by shifting them to these cheaper open source tokens, and that's just more compute. So a company getting smarter about which model to use for which task, that may lead to a stabilization in their spend or even a decline, but it actually has nothing to do with the amount of GPU compute hours they are effectively consuming behind these model layers of this router. The GPU compute hours probably are going up as you shift to these cheaper tokens you can use more of.
00:29:58 · Gavin Baker, Patrick O'Shaughnessy
That's happening to like a cutting edge of public cloud Companies, and then you have this whole wave of AI natives, they're leading it to this so hard, and they're not hiring humans, they're just putting it mostly into tokens, they're not slowing down. And then you have companies on the east coast of America who have like barely adopted AI, companies broadly speaking, you know, not in the coast, who maybe aren't as cutting, and then Europe, who's just trying to figure out how to regulate AI, you [00:30:30] S0: know, [00:30:31] S2: before [00:30:32] S0: using it. So just
00:30:33 · Gavin Baker
[00:30:33] S2: like there's kind of these differential waves of adoption all happening at the same time, but the thought I can't get out of my mind is like, I think I said it maybe last time, but just y'all access it or not, like five hundred thousand people in the world, two hundred fifty thousand maybe are using agentic AI, and we're in an acute compute shortage. There's seven or eight billion people on the planet. What happens when we go from five hundred thousand to one percent to a hundred million, you know, to five hundred million? It is interesting, you know, a lot of people, I do think it's like helpful to post on X to see the pushback and a lot of people are saying, "We accept your argument that hyperscalers are a dirty, get us compute reprices, their operating cash flows gonna accelerate, and maybe we can fund this, but like, where's that operating cash flow gonna come from? Where is the customer?" And kind of definitionally, it has to either come from faster economic growth through productivity, kind of Satya's comments, like "Either we're gonna start growing ten percent or we're not." Or labor substitution. And for sure, I think in a lot of these AI natives, you're seeing labor substitution, but not because they're firing people, they're just not hiring nearly as many humans. The gross profit dollars per FTE in A16Z, ICANN, a bunch of companies that have done this work, they're vertical, particularly relative to past generations of startups. And then it is interesting, are you doing any surveys of your companies and their token spend relative to labor spend?
00:32:05 · Patrick O'Shaughnessy, Gavin Baker
[00:32:05] S0: Oh yeah, I mean, it's tokens as a percent of total comp spend or something like that. And what, what are the ranges you've seen? I mean, like in the really pilled companies, like it gets really high, twenty percent, twenty-five percent. [00:32:17] S2: Our friend Nile Patel at his, at his company, he's an ASI backseat, but he's at thirty percent. [00:32:23] S0: That's probably the [00:32:24] S2: highest one I've heard. I've actually heard a fifty, and there's twenty-five trillion dollars in knowledge work Let's take your twenty percent number, that's five trillion, and that either comes out of labor substitution or faster economic growth, and we really, really, really want as humans to come from faster economic growth. [00:32:45] S0: One interesting thing I heard this morning from one of the great leading technology CEOs that's founded several companies, if you look at the founder-led and controlled companies and adjust for some of the like COVID-era over-hiring, nobody's really laying people off. These are the people that would probably be most quick to adopt AI to become more efficient or whatever, like they're not really doing jack-assed like huge-scale layoffs, which probably tells you something about where they think there will be lots of opportunity to still have people plus--
00:33:12 · Gavin Baker
[00:33:12] S2: A hundred percent. Well, the bull case, you've- I've seen charts from Cognition, Rayop, and Stripe that the companies that are spending the most on AI are growing meaningfully faster. Yeah, I love that Cognition index. Yeah, the Cognition index is wild. All the skeptics will point out rightfully, it's not really controlling for industry, but then if like you dig down into it, I think one of them gave an example of, I forget if it was a plumber or an HVAC contractor, but like everybody who's a blue collar worker is doing great 'cause of AI. By the way, something that I think we should touch on These LTAs. So everything's in a shortage. If there's weakness, it's just 'cause we can't energize the gigawatts fast enough. The gigawatts are gonna get energized like regulatory policies moving in a good way, the turbine manufacturers, the diesel genset manufacturers, you know, you're ripping turbines off old airplanes and, you know, reconditioning them and then repurposing them. There's crazy things happening. Capitalism's very, very good at this. But I do think one of the most important questions in the market, and like a transition in the market that I got wrong, is we are shifting, particularly for memory more than anything else, from crushing numbers in the short term to they are trading short term upside for these, what they call supply chain agreements, long term agreements, LTAs. There's many flavors, but customer pre-pays, there's a floor and a ceiling. And this comes back to the point about labor, because, you know, a lot of people after firing too many people during COVID were really reluctant to lay people off. They talked about labor hoarding, if you remember a few years ago. You remember this? Yes. Let's just think about the game theory of breaking an LTA. So there's four companies that matter at scale. There's Amazon with their Trademms, there's Google with their TPUs, there's AMD, and then there's Nvidia, who's like much bigger than everybody else combined. Let's just say it's two thousand and twenty-seven, and it's very important to realize memory is, the more memory you put with Flop for a given unit of compute, the more tokens you get out.
00:35:22 · Gavin Baker
It's the single most important thing you could do to increase token output per unit of compute, and then that obviously definitionally actually lowers costs, which is why the demand hasn't responded at all negatively. There's been no elasticity just because it's the access that is dominating all others. And this is at some level like a giant Game of Thrones or Ippar's between these companies. Okay, it's two thousand and twenty-seven or twenty-eight. You're vaguely tempted to break one of these LTAs and try and get a lower price. But to a large degree, market shares, I think for the next several years, are going to be determined by supply chain allocations and kind of what you have pre-purchased. So if you break the LTA, this is assuming we're not in a severe oversupply situation, the game theory even holds in a severe oversupply situation, if you break your LTA, and then in the next Two or three years, for any reason, leverage shifts back to the Bibury guys, you're out of business. It's over. Let's just say Google breaks an LTA, there's an oversupply, I'm making this up at twenty-eight, twenty-nine, they break their LTAs. Well, if they're breaking their LTAs, it probably means, you know, your oversupply, prices are coming down, and then, you know, capacity naturally contracts. Well, what do you think's gonna happen to Google's allocations? And then, you know, this is a cyclical industry, and oversupply is followed by undersupply. What do you think they think is gonna happen to their allocations next time? So I just think given that this is the axis around which kind of everything is revolving, you might blow up your entire business and your franchise by breaking an LTA, and that was never the case before. Apple, who cares? They don't have a competitor, they're overwhelmingly the largest purchaser. This is, you know, going back three, four, five years, they know they can do whatever they want with no consequences, 'cause their volume is so big that even if they like super screw Heineken, Micron will of course- Take them. This is just different, you know? You have at least four players, didn't you have all the startups, your investor and Etched. If you break an LTA, they just say, "Okay, fine, great, you broke the price agreement, we're gonna break the volume agreement, and screw you, we're gonna give the volume to your competitor.