Decoder with Nilay Patel
00:33:23 · Nilay Patel, S3
[00:33:23] S2: This... But this is my prevention versus cure. Right? The the prevention is cheap, but it's all loss on the p and l statement. Right? It's all just cost. Yeah. If the bad thing happens, like, well, we have to do anything to fix this because there's economic benefit to having a functional business again. And it seems like in this moment, even though... So sort of like basic understanding of that risk versus cost is totally scrambled and no one has quite figured it out yet. [00:33:48] S3: Yeah. I totally agree. So, like, I'll give you a a very practical example of of of a way CISOs and their bosses are going to have to change on this. Every company in the next... I was gonna say six months, but really in the next ninety days is going to have to RTX themselves. You're just going to have You're gonna [00:34:05] S2: have to run the attack on yourself.
00:34:06 · S3
[00:34:06] S3: Yes. If not with real RTX, you're going to have to use a AI red teaming product that is functional equivalent, or you're going to have to hire a company to do that for yourself. If you do not, you will get breached. Yeah. It just... It That's terrifying. Right? Because these things are completely nondeterministic by definition. They are very smart. They are very fast. They are so fast that no matter what... If you hire, like, an AI pen test, AI red team company, they will promise you verbally, oh, it is completely safe. And then they will give you a contract that says, our product will give you cancer, will emanatize the eschaton, and will likely end the human race, and you hold us harmless when any of these things happen. Right? So there is a disconnect between what they will say verbally to you and what their lawyers have written. Right? Because that company has, by definition, only existed for six months or at least the product has existed for six months if it is an older company. They have no idea what their product is going to do when it meets your your software. They have no idea whatever they say to you, but you have to do it because some dude is going to do it to you, and you need to know what is going to happen and and fix it because at least maybe those people will break your system. They might cause downtime, but at least what they're not going to do is steal your data and ransom it or intentionally shut down your blood bank refrigerators. That is what happened in France. Right? So that is exactly the kind of thing that a CISO... A conservative CISO would never even ask to do, and a conservative company, even if they have a risk taking CISO, would never let their CISO do. And so what they're... What's gonna happen next six months? A ton of companies are gonna get breached by some 19 year old running RTX and then buying tokens off OpenRouter or running it locally on their gaming PC. Right? They're gonna turn off Call of Duty, run RTX, then turn Call of Duty back on. Right? And while they did that, they broke into a bank who their policy said, oh, we can never red team ourselves. Or they said yes, but they ended up in a thirty day negotiation because their lawyers are like, I want to have... You have to have a $100,000,000 in insurance, and we wanna have, you know, this kind of...
00:36:21 · S3, S4
Like, they say yes, but their vendor risk management process and their legal process is so onerous. There's no possible way they could actually accept the risk. That is my message also. You are going to have to take that risk to actively red team yourself, to front run the bad guys who are going to be... Who are doing that right now, right now, as we speak, and as people are listening to this twenty four hours from now, you are possibly have an AI system who is trying every single API endpoint and website that you pass... You might not know you have because it was like a marketing site that was set up a year ago for an for an executive event, and is gonna find a bug and get in, and then it's gonna move incredibly fast through your network. So you'd rather pay somebody or have your own team do that. You need to accept that risk. And almost no company larger than a couple 100 people has a risk management framework that allow them to do that. Ad[00:37:25] S4: Support for the show comes from CrowdStrike. Here on Decoder, we're peering beyond the traditional tech realm and into all the places AI is shaking things Cybersecurity is key to that conversation. CrowdStrike gets that. That's why AI detection and response or AIDR is emerging as a new category of cybersecurity for the AI era. It helps organizations see, monitor, and secure AI across the enterprise. CrowdStrike. We secure AI. We stop breaches. Go to crowdstrike.com/decoder to learn more.
00:38:09 · Nilay Patel, S3
[00:38:09] S2: There are there are several costs associated with this. And one, I think, is just like the most common in business, which is, well, if we find out there's a problem, we'll have to pay to fix it. So what if we just wait? Right? Like, you can see that coming. What if we find out that we are catastrophically in trouble? And then there's how much will it cost to fix it? And then there's what will we do to fix it? And the argument that I've heard about what will we do to fix it is, well, we'll just unleash a swarm of defensive AI tools that are watching the networks and watching the code. And they'll make a bunch of autonomous decisions, and they'll fight the autonomous script kiddies who are armed with RTX on their Mac studios. And something about that argument in my head. I've heard, for example, Palo Alto Networks CEO, Nikesh has made this argument. Right? You need you need to give the defensive agent some autonomous capabilities. And I'm like, well, hold on. Like, we... OpenAI can't control the swarm that went and attacked Hugging Face. How would I control the defensive agent from doing something equally damaging to prevent an attack? And I I don't think that there's a technical solution there that makes it clear what to do once you found all of the vulnerabilities. [00:39:16] S3: Couple things. One [00:39:18] S2: I told you I was gonna only ask dumb questions. I just wanna No.
00:39:18 · Nilay Patel, S3
[00:39:18] S2: I told you I was gonna only ask dumb questions. I just wanna No. [00:39:21] S3: Okay. That is a fantastic question. You have not asked a dumb question yet. I'm I'm honest. I... You have not asked a dumb question yet. That's a fantastic question. Okay. I'm gonna give you a shorthand that you can use going forward. If somebody actually uses the term swarm, then [00:39:35] S2: I don't know if... I mean, to be clear, I don't know if Nikesh used the word swarm.
00:39:38 · S3
[00:39:38] S3: Okay. I'm not saying he did. Yeah. And I have great respect for Nikesh. He is one of the best businessmen in in all of security. So I... This is not on him, be... But Swarm is used all the time. Yeah. But when I hear somebody use Swarm, I immediately discount all of their technical opinions by about 80%. The... It is going to become... It is already, and it will become the most overused term in all of security. Okay. The actual, quote, unquote, swarm from OpenAI was something between a different time, 700 to 1,400 agents. That had to run on... It it... It's hard to tell because we don't have the actual, like, size of the models, but, you know, 300,000,000 to almost $1,000,000,000 in hardware. That doesn't include the actual data center cost. It... The actual hugging face attack was... I I did some back of the envelope map... Math. It may be a million dollars in electricity. Right? No attacker is going to do that. And and in the end, that was stupid because these things were actually trapped. They were not prompted to attack Hugging Face. Most of them were doing something totally different and were kind of enlisted by the one thing that actually wanted to get to Hugging Face. They used a tiny fraction of their token budget to help their little friend, and then they had to tunnel their way out. So it was the most inefficient way to possibly attack Hugging Face. Almost nobody is going to be attacked by a swarm. So if somebody says swarm, they do not actually understand at a low... At the bottom technical level what's going on. The RTX attacks are almost certainly one single agent, maybe two.