"The Cognitive Revolution"
00:36:46 · Mike San Miguel, Prakash Narayanan
[00:36:46] S2: No. Typically, what happens is last minute, you'll have a project push. And, that's why some of these, OEMs will reserve the right to resell inventory. If, let's say, one of their hyperscalers sites isn't ready, they'll be able to sell that inventory and then backfill it when that data center gets to be ready. Typically, have to have an address. [00:37:07] S1: At some point, we will see these h 100 rental prices dip. And now it sounds as though if the prices go down, you're going to see more retirement. So you're going to see more of those enter the trading market. Are there any places to deploy them?
00:37:23 · Mike San Miguel
[00:37:23] S2: Yeah, great question. So yes, I agree that as these rates go down, you'll see more that get, decommissioned. In terms of sites to deploy them, there will be. Part of it is that not every site can operate every GPU. Vera Rubin is strictly a liquid cooled solution. That's NVIDIA's next generation. Right? The cement slabs that those go on have to be reinforced. Typically, those are what? Like a ton per rack. The weight is insane. The power draw is about 230 kilowatts per rack. To give you some comparison, for two b three hundreds, you're looking at, you know, 30, 35 kilowatts, per rack. Not every site can do Vera Rubin. Not every site can do g v 300, which is similar to Vera Rubin on power. So, you know, it'll make sense to convert some sites. It won't make sense to convert other sites. AI is not every single workload. Right? AI is a family of different workloads. All of them have different requirements. There is some sizing differences right now, but it's mostly RAM and shared storage and local storage and, some networking back end. But we we need to get more efficient with how we size to the workload. Because the whole point of this hardware isn't to own the hardware or to support NVIDIA's, you know, next, you know, what, trillion dollar milestone. The point is to run a specific workload that has real world implications for end users and helps create new businesses. So, you know, as we get out of these very hardware constrained environments, we're able to look more at the software and then the hardware gets more efficient.
00:39:11 · Nathan Labenz, Prakash Narayanan, Mike San Miguel
[00:39:11] S0: Part four, who's checking? Mike San Miguel on the rules that travel with the hardware. [00:39:20] S1: Do they do they do a KYC of who is the person buying the thing or who owns the person buying the thing? Do they do they do do they do checks on that kind of stuff? [00:39:30] S2: Yeah. Yeah. And actually, part of my role here was was spinning up our our our KYC department. A lot of the OEMs will not, even quote without KYC being done. Some will not ship without it being done. Internally, you know, we basically do not accept projects without KYC being done except in some rare circumstances where it's big names or or companies that we've worked with for years. There are limits on, you know, what percentage can be owned by China. [00:40:02] S1: What what percentage of that company can be owned by China? Mhmm. [00:40:06] S2: Yeah. Like, I think they might be going down to 49%. I know typically 50% or more. You you can't sell to companies that are on 50% by by a Chinese entity. [00:40:19] S1: Do do do they do this... A lot of Chinese entities will go through a Cayman Islands or a BVI or whatever. But do they do they trace back through that?
00:40:19 · Prakash Narayanan, Mike San Miguel, Nathan Labenz
[00:40:19] S1: Do do do they do this... A lot of Chinese entities will go through a Cayman Islands or a BVI or whatever. But do they do they trace back through that? [00:40:28] S2: Yeah. Yeah. They do. I mean, at least it's something we do. Every company is a little bit different, but we we do international orders. Right? We're not just, geo bound in North America. So, for us, you know, we're we're gonna trace back not just to the hardware purchaser is, but also if they have an off take. They're signing, an agreement with a neo cloud. You cannot legally sell this equipment with a off taker being a Chinese entity, for instance. [00:41:03] S0: Mike signed off there. Prakash and I kept talking. [00:41:09] S1: We talked to people like Samuel Hammond and others on the DC facing national security decision makers. But then you go down to the deployment guys, and these guys are running around, they're getting permissions, and every six months they're expecting an Nvidia checkup, and they're chasing on KYC, and they even know they're going to the owners, and they're going to the users, and maybe they don't know the user of the user. They're saying, Okay, maybe that is unclear. So I I I I wonder to what extent the policy discussions at the top end are divorced from really how in-depth the execution is at this point. Yeah.
00:41:59 · Nathan Labenz, Prakash Narayanan
[00:41:59] S0: I bet everything everywhere all at once is probably true here because a lot of what he described did sound to me kind of like the fracking boom in the sense that there's just, like, a lot of... I mean, it's capitalism at work. Right? I mean, if there's a lot of things coming together all at the same time, there's a ton of moving pieces in everything from the sites and the approvals and the local politics and the grid and, you know, and, of course, the chips. And you you gotta reinforce the concrete and, you know, not every site is ready to take on the the latest super heavy racks. I mean, just so many little details, so many people coming in together to do their part to create something that at the end of the day, at the consumer level is just like, you know, gives you some appreciation for why Claude occasionally has some downtime. Like, in general, you know, it's pretty amazing how smooth it all is. So I could see both, you know, the enforcement could, in many cases, be pretty legit, especially [00:43:09] S1: among,
00:43:09 · Prakash Narayanan, Nathan Labenz
[00:43:09] S1: among, [00:43:09] S0: you know, people who care about it or, you know, who are, just conservative enough that they wanna make sure they're absolutely on the right side of the law. And at the same time, that sounded chaotic enough that, if you told me that, like, I don't know, you know, 20% of the GPUs, like, somehow don't get their regular NVIDIA six month check-in, and of those, you know, some percentage found their way to wherever, that wouldn't shock me at all, you know, based on on what he said. It sounds like there's, you know, eighty twenty is obviously a simplistic way to describe it, but it sounds like there's a lot of sincere effort going in and then also just enough noise and and chaos that probably... The the opportunity for things to slip through is also pretty substantial, I would bet. Then Prakash turned to China and the pressure on American companies over their Chinese customers.