"The Cognitive Revolution"
00:33:16 · Mike San Miguel, Prakash Narayanan, Nathan Labenz
[00:33:16] S2: Mostly for b 300, it's new. Occasionally, we'll see abandoned orders. They... They're new enough and they're profitable enough renting out that typically we we really haven't seen anything used hit the market yet. [00:33:29] S1: And how much does one of those cost? Like like, off the top of your head, like, you know, right now asking asking price or clearing, like, price right now? [00:33:37] S2: Yeah. Depending on on the specifications, the OEM warranty terms, of that, you're looking anywhere from $5.50 to $6.50. Typically, for bulk orders, it's on the lower end. [00:33:51] S1: And and people are trading new. And what what is the typical lot size? Are people trading like, okay. One one rack system or, 10 rack systems or 100 rack systems or, you know [00:34:00] S2: Yeah. Yeah. Well, the the units per rack come down to the data center configurations because that's about how much power can be delivered to each rack. Typically, most data centers will be able to operate two per rack right now unless you've got a really nice dense supporting DLC data center. In terms of sizes, we trade everything from one or two units up to about a 128 or two fifty six. So Wow. Yeah. [00:34:29] S1: $50,000,000. $50,000,000 trades. [00:34:32] S2: Yeah. They get they get pretty big. The numbers are pretty pretty aggressive, but it's, you know, it's the scale of the build out that's happening. [00:34:40] S0: Prakash asked whether those prices track the hourly rental indices.
00:34:40 · Nathan Labenz, Mike San Miguel, Prakash Narayanan
[00:34:40] S0: Prakash asked whether those prices track the hourly rental indices. [00:34:46] S2: So, you know, one of the areas that we we help our customers with understanding is this difference between a contracted rate and an hourly rental rate on these GPUs. So if you look at, like, an AI factory contract rental for b 300, typical, you know, market price right now is $4.20 to $4.50 an hour depending on the market that you're in and the configuration. Now if you're only getting, you know, $4.20 per GPU per hour, that's a very different rate from what you're gonna see on those indices. That's eight to nine out, dollars per hour per GPU. So if you have the the capital to go out and buy, you know, lower amount of nodes, deploy them, then, you know, typically, you can get a better rate of return as opposed to buying, like, full AI factory and negotiating an offtake contract with Neo Cloud. [00:35:40] S1: Why is there excess inventory? Like, I mean, these things are pretty expensive. I I I assume, like, in order to buy them, you have to put on a prepayment or something like that before before they start manufacturing them for you.
00:35:40 · Prakash Narayanan, Mike San Miguel
[00:35:40] S1: Why is there excess inventory? Like, I mean, these things are pretty expensive. I I I assume, like, in order to buy them, you have to put on a prepayment or something like that before before they start manufacturing them for you. [00:35:51] S2: Yeah. You do. Part of it has to do with data center availability. So, this market's looked like a series of different bottlenecks. Right now, it's liquid cooled and data center readiness. So that number I gave you with most data centers being able to support, let's say, two nodes per rack, that has to do with constraints, not on the actual hardware side, but on the data center's power delivery. Same with sort of the cooling build out as well as sort of a bottleneck right now. Liquid cooled, we're starting to see data center inventory coming available January, q one. But most of that's already sold out. There's a lot of data centers that are now telling us it's q three twenty twenty seven or 2028. [00:36:39] S1: Are are people are people trading the market? Are they buying are they buying their systems without having places to plug them into? [00:36:46] S2: No. Typically, what happens is last minute, you'll have a project push. And, that's why some of these, OEMs will reserve the right to resell inventory. If, let's say, one of their hyperscalers sites isn't ready, they'll be able to sell that inventory and then backfill it when that data center gets to be ready. Typically, have to have an address.
00:36:46 · Mike San Miguel, Prakash Narayanan
[00:36:46] S2: No. Typically, what happens is last minute, you'll have a project push. And, that's why some of these, OEMs will reserve the right to resell inventory. If, let's say, one of their hyperscalers sites isn't ready, they'll be able to sell that inventory and then backfill it when that data center gets to be ready. Typically, have to have an address. [00:37:07] S1: At some point, we will see these h 100 rental prices dip. And now it sounds as though if the prices go down, you're going to see more retirement. So you're going to see more of those enter the trading market. Are there any places to deploy them?
00:37:23 · Mike San Miguel
[00:37:23] S2: Yeah, great question. So yes, I agree that as these rates go down, you'll see more that get, decommissioned. In terms of sites to deploy them, there will be. Part of it is that not every site can operate every GPU. Vera Rubin is strictly a liquid cooled solution. That's NVIDIA's next generation. Right? The cement slabs that those go on have to be reinforced. Typically, those are what? Like a ton per rack. The weight is insane. The power draw is about 230 kilowatts per rack. To give you some comparison, for two b three hundreds, you're looking at, you know, 30, 35 kilowatts, per rack. Not every site can do Vera Rubin. Not every site can do g v 300, which is similar to Vera Rubin on power. So, you know, it'll make sense to convert some sites. It won't make sense to convert other sites. AI is not every single workload. Right? AI is a family of different workloads. All of them have different requirements. There is some sizing differences right now, but it's mostly RAM and shared storage and local storage and, some networking back end. But we we need to get more efficient with how we size to the workload. Because the whole point of this hardware isn't to own the hardware or to support NVIDIA's, you know, next, you know, what, trillion dollar milestone. The point is to run a specific workload that has real world implications for end users and helps create new businesses. So, you know, as we get out of these very hardware constrained environments, we're able to look more at the software and then the hardware gets more efficient.
00:39:11 · Nathan Labenz, Prakash Narayanan, Mike San Miguel
[00:39:11] S0: Part four, who's checking? Mike San Miguel on the rules that travel with the hardware. [00:39:20] S1: Do they do they do a KYC of who is the person buying the thing or who owns the person buying the thing? Do they do they do do they do checks on that kind of stuff? [00:39:30] S2: Yeah. Yeah. And actually, part of my role here was was spinning up our our our KYC department. A lot of the OEMs will not, even quote without KYC being done. Some will not ship without it being done. Internally, you know, we basically do not accept projects without KYC being done except in some rare circumstances where it's big names or or companies that we've worked with for years. There are limits on, you know, what percentage can be owned by China. [00:40:02] S1: What what percentage of that company can be owned by China? Mhmm. [00:40:06] S2: Yeah. Like, I think they might be going down to 49%. I know typically 50% or more. You you can't sell to companies that are on 50% by by a Chinese entity. [00:40:19] S1: Do do do they do this... A lot of Chinese entities will go through a Cayman Islands or a BVI or whatever. But do they do they trace back through that?