"The Cognitive Revolution"
00:30:00 · Nathan Labenz
[00:30:00] S0: But at the same time, we do have these ballot initiatives. And, like, 40 ballot initiatives, from what I had read a little bit, is, like, one of the bigger waves of ballot initiatives. Like, these these sorts of things only come around when you have something like gay marriage, you know, was one where there was, like, a whole wave of initiatives. There was a a similar wave in terms of affirming or protecting abortion rights in the in the wake of the Roe v Wade, overturning. And so this does seem like there... And what what I'd like about ballad is I've long been a big direct democracy fan and advocate. Not to say that people never make mistakes, but it is something where you can unbundle your concerns about a particular issue from the broader political conversation. And the fact that we did see so much of that and that they all won, I guess, with one exception, does to me seem like a sign of things to come. And I would... You know, we'll see. Maybe we'll get the companies like cutting people in on the deal. I think, you know, we've talked about that a few times as well, and I I... People should get paid. I definitely am on board with that. But doesn't it seem like there is, like, a pretty real signal there to you that, like, yes, it's not the biggest thing, but it it does seem like there's a a pretty serious groundswell of people willing to work, you know, willing to go collect signatures. That stuff is not easy. You gotta stand on street corners and, you know, get strangers to sign petitions. And then it still has to pass on top of that. So the fact that so many of those were organized and so many of them passed does seem like a big signal of what we can expect to me.
00:31:48 · Prakash Narayanan, Nathan Labenz, Mike San Miguel
[00:31:48] S1: I agree, but I'm like, why? Why did they sign? Right? And is... Do you think they signed because it's the end of the world? [00:31:58] S0: No. No. Yeah, think your diagnosis was pretty right. [00:32:01] S1: Right. So I think the outcome of this is that if you're going to be Google or Meta, you're going to be paying Google or Meta prices for everything, for the land, for the labor, for the construction. [00:32:18] S0: Our Thursday guest was Mike San Miguel of Luxor Technology, which came up in Bitcoin mining and now trades new and refurbished GPU servers. When he says $5.50 to six fifty, he means 550,000 to $650,000 for one eight GPU server. [00:32:35] S1: So what what is what is trading frequently? [00:32:38] S2: Right now, it's mostly b three hundreds. [00:32:41] S1: Three hundreds are trading frequently. [00:32:43] S2: Yeah. [00:32:43] S1: And and do they trade as a chip or as a node or as a as a rack as eight eight eight together? Or how how does that work? [00:32:51] S2: Yeah. Great question. So typically for these servers, it's an HGX system. So eight GPUs per server, validated build built by an eight OEM. So you've got the HGX baseboard, dual CPUs, 32 sticks of RAM, depending on the size RAM, shared storage, east to west NICs. [00:33:14] S1: Do people trade used on you right now? [00:33:16] S2: Mostly for b 300, it's new. Occasionally, we'll see abandoned orders. They... They're new enough and they're profitable enough renting out that typically we we really haven't seen anything used hit the market yet.
00:33:16 · Mike San Miguel, Prakash Narayanan, Nathan Labenz
[00:33:16] S2: Mostly for b 300, it's new. Occasionally, we'll see abandoned orders. They... They're new enough and they're profitable enough renting out that typically we we really haven't seen anything used hit the market yet. [00:33:29] S1: And how much does one of those cost? Like like, off the top of your head, like, you know, right now asking asking price or clearing, like, price right now? [00:33:37] S2: Yeah. Depending on on the specifications, the OEM warranty terms, of that, you're looking anywhere from $5.50 to $6.50. Typically, for bulk orders, it's on the lower end. [00:33:51] S1: And and people are trading new. And what what is the typical lot size? Are people trading like, okay. One one rack system or, 10 rack systems or 100 rack systems or, you know [00:34:00] S2: Yeah. Yeah. Well, the the units per rack come down to the data center configurations because that's about how much power can be delivered to each rack. Typically, most data centers will be able to operate two per rack right now unless you've got a really nice dense supporting DLC data center. In terms of sizes, we trade everything from one or two units up to about a 128 or two fifty six. So Wow. Yeah. [00:34:29] S1: $50,000,000. $50,000,000 trades. [00:34:32] S2: Yeah. They get they get pretty big. The numbers are pretty pretty aggressive, but it's, you know, it's the scale of the build out that's happening. [00:34:40] S0: Prakash asked whether those prices track the hourly rental indices.
00:34:40 · Nathan Labenz, Mike San Miguel, Prakash Narayanan
[00:34:40] S0: Prakash asked whether those prices track the hourly rental indices. [00:34:46] S2: So, you know, one of the areas that we we help our customers with understanding is this difference between a contracted rate and an hourly rental rate on these GPUs. So if you look at, like, an AI factory contract rental for b 300, typical, you know, market price right now is $4.20 to $4.50 an hour depending on the market that you're in and the configuration. Now if you're only getting, you know, $4.20 per GPU per hour, that's a very different rate from what you're gonna see on those indices. That's eight to nine out, dollars per hour per GPU. So if you have the the capital to go out and buy, you know, lower amount of nodes, deploy them, then, you know, typically, you can get a better rate of return as opposed to buying, like, full AI factory and negotiating an offtake contract with Neo Cloud. [00:35:40] S1: Why is there excess inventory? Like, I mean, these things are pretty expensive. I I I assume, like, in order to buy them, you have to put on a prepayment or something like that before before they start manufacturing them for you.
00:35:40 · Prakash Narayanan, Mike San Miguel
[00:35:40] S1: Why is there excess inventory? Like, I mean, these things are pretty expensive. I I I assume, like, in order to buy them, you have to put on a prepayment or something like that before before they start manufacturing them for you. [00:35:51] S2: Yeah. You do. Part of it has to do with data center availability. So, this market's looked like a series of different bottlenecks. Right now, it's liquid cooled and data center readiness. So that number I gave you with most data centers being able to support, let's say, two nodes per rack, that has to do with constraints, not on the actual hardware side, but on the data center's power delivery. Same with sort of the cooling build out as well as sort of a bottleneck right now. Liquid cooled, we're starting to see data center inventory coming available January, q one. But most of that's already sold out. There's a lot of data centers that are now telling us it's q three twenty twenty seven or 2028. [00:36:39] S1: Are are people are people trading the market? Are they buying are they buying their systems without having places to plug them into? [00:36:46] S2: No. Typically, what happens is last minute, you'll have a project push. And, that's why some of these, OEMs will reserve the right to resell inventory. If, let's say, one of their hyperscalers sites isn't ready, they'll be able to sell that inventory and then backfill it when that data center gets to be ready. Typically, have to have an address.
00:36:46 · Mike San Miguel, Prakash Narayanan
[00:36:46] S2: No. Typically, what happens is last minute, you'll have a project push. And, that's why some of these, OEMs will reserve the right to resell inventory. If, let's say, one of their hyperscalers sites isn't ready, they'll be able to sell that inventory and then backfill it when that data center gets to be ready. Typically, have to have an address. [00:37:07] S1: At some point, we will see these h 100 rental prices dip. And now it sounds as though if the prices go down, you're going to see more retirement. So you're going to see more of those enter the trading market. Are there any places to deploy them?