"The Cognitive Revolution"
00:37:23 · Mike San Miguel
[00:37:23] S2: Yeah, great question. So yes, I agree that as these rates go down, you'll see more that get, decommissioned. In terms of sites to deploy them, there will be. Part of it is that not every site can operate every GPU. Vera Rubin is strictly a liquid cooled solution. That's NVIDIA's next generation. Right? The cement slabs that those go on have to be reinforced. Typically, those are what? Like a ton per rack. The weight is insane. The power draw is about 230 kilowatts per rack. To give you some comparison, for two b three hundreds, you're looking at, you know, 30, 35 kilowatts, per rack. Not every site can do Vera Rubin. Not every site can do g v 300, which is similar to Vera Rubin on power. So, you know, it'll make sense to convert some sites. It won't make sense to convert other sites. AI is not every single workload. Right? AI is a family of different workloads. All of them have different requirements. There is some sizing differences right now, but it's mostly RAM and shared storage and local storage and, some networking back end. But we we need to get more efficient with how we size to the workload. Because the whole point of this hardware isn't to own the hardware or to support NVIDIA's, you know, next, you know, what, trillion dollar milestone. The point is to run a specific workload that has real world implications for end users and helps create new businesses. So, you know, as we get out of these very hardware constrained environments, we're able to look more at the software and then the hardware gets more efficient.
00:39:11 · Nathan Labenz, Prakash Narayanan, Mike San Miguel
[00:39:11] S0: Part four, who's checking? Mike San Miguel on the rules that travel with the hardware. [00:39:20] S1: Do they do they do a KYC of who is the person buying the thing or who owns the person buying the thing? Do they do they do do they do checks on that kind of stuff? [00:39:30] S2: Yeah. Yeah. And actually, part of my role here was was spinning up our our our KYC department. A lot of the OEMs will not, even quote without KYC being done. Some will not ship without it being done. Internally, you know, we basically do not accept projects without KYC being done except in some rare circumstances where it's big names or or companies that we've worked with for years. There are limits on, you know, what percentage can be owned by China. [00:40:02] S1: What what percentage of that company can be owned by China? Mhmm. [00:40:06] S2: Yeah. Like, I think they might be going down to 49%. I know typically 50% or more. You you can't sell to companies that are on 50% by by a Chinese entity. [00:40:19] S1: Do do do they do this... A lot of Chinese entities will go through a Cayman Islands or a BVI or whatever. But do they do they trace back through that?
00:40:19 · Prakash Narayanan, Mike San Miguel, Nathan Labenz
[00:40:19] S1: Do do do they do this... A lot of Chinese entities will go through a Cayman Islands or a BVI or whatever. But do they do they trace back through that? [00:40:28] S2: Yeah. Yeah. They do. I mean, at least it's something we do. Every company is a little bit different, but we we do international orders. Right? We're not just, geo bound in North America. So, for us, you know, we're we're gonna trace back not just to the hardware purchaser is, but also if they have an off take. They're signing, an agreement with a neo cloud. You cannot legally sell this equipment with a off taker being a Chinese entity, for instance. [00:41:03] S0: Mike signed off there. Prakash and I kept talking. [00:41:09] S1: We talked to people like Samuel Hammond and others on the DC facing national security decision makers. But then you go down to the deployment guys, and these guys are running around, they're getting permissions, and every six months they're expecting an Nvidia checkup, and they're chasing on KYC, and they even know they're going to the owners, and they're going to the users, and maybe they don't know the user of the user. They're saying, Okay, maybe that is unclear. So I I I I wonder to what extent the policy discussions at the top end are divorced from really how in-depth the execution is at this point. Yeah.
00:41:59 · Nathan Labenz, Prakash Narayanan
[00:41:59] S0: I bet everything everywhere all at once is probably true here because a lot of what he described did sound to me kind of like the fracking boom in the sense that there's just, like, a lot of... I mean, it's capitalism at work. Right? I mean, if there's a lot of things coming together all at the same time, there's a ton of moving pieces in everything from the sites and the approvals and the local politics and the grid and, you know, and, of course, the chips. And you you gotta reinforce the concrete and, you know, not every site is ready to take on the the latest super heavy racks. I mean, just so many little details, so many people coming in together to do their part to create something that at the end of the day, at the consumer level is just like, you know, gives you some appreciation for why Claude occasionally has some downtime. Like, in general, you know, it's pretty amazing how smooth it all is. So I could see both, you know, the enforcement could, in many cases, be pretty legit, especially [00:43:09] S1: among,
00:43:09 · Prakash Narayanan, Nathan Labenz
[00:43:09] S1: among, [00:43:09] S0: you know, people who care about it or, you know, who are, just conservative enough that they wanna make sure they're absolutely on the right side of the law. And at the same time, that sounded chaotic enough that, if you told me that, like, I don't know, you know, 20% of the GPUs, like, somehow don't get their regular NVIDIA six month check-in, and of those, you know, some percentage found their way to wherever, that wouldn't shock me at all, you know, based on on what he said. It sounds like there's, you know, eighty twenty is obviously a simplistic way to describe it, but it sounds like there's a lot of sincere effort going in and then also just enough noise and and chaos that probably... The the opportunity for things to slip through is also pretty substantial, I would bet. Then Prakash turned to China and the pressure on American companies over their Chinese customers.
00:44:10 · Prakash Narayanan
[00:44:10] S1: Like, for example, Oracle selling capacity to Tencent, $7,000,000,000 of capacity, that was canceled. Right? So the national security infrastructure is pressuring these companies to cancel that. It's an American company doing business, it's outside China, and not sending chips into China. And at the same time, there is no pushback on the political infrastructure that is allowing this. Why don't you just make laws? Why pressure the companies? I just don't like it. If you wanna make laws, just make laws. But if you don't have the political willpower to make those laws, and you try to slide this under the gate, like why? You have a mandate now, right? You have a mandate in Congress, like do something, right? We say compute is the new oil, right? And so there is also this impact on compute. Let's say you decide not to sell compute to China, like zero, zero Chinese content, right? Do you also mean that YouTube cannot show AI generated video to Chinese participants, to Chinese viewers, to Chinese customers? Because that is an AI product. And Google needs to generate that AI product. Google has to use compute. And that compute is gonna be used on behalf of a Chinese customer. You can restrict it at the data center level. You can restrict it at the raw compute level. You can restrict it at the product of compute level. But at some level, you're going to end up selling to the Chinese because they are global consumers. Right?