Decoder with Nilay Patel
00:02:37 · S3
[00:02:37] S3: October is, you know, traditionally security awareness month. Right? Like, lots of companies run events and and deal both internally and then tell people, you know, change your password. Use multiple passwords. Use a password manager. And and what has happened over the last month is we've had, you know, a couple of these big bugs, PHP, Node, and such. What's been less discussed is a huge number of companies have been reached. You have a series of AI related companies, the Neo Clouds, that have been breached in succession. You have the Binance hack, which is the largest heist of anything in human history. You have the cursor incident where, you know, one of the things that's going on here is these hacks are being chained together. So it looks like from what we know right now, one of the companies that was broken into was Fireworks, which is a prominent provider of AI services to a number of companies. From the Fireworks hack, API keys were stolen that allowed people to break into Cursor. Once they got into Cursor, they then pushed a bad plug in, which went to 5,000 more companies, and that caused 5,000 potential breaches. So it's just this kind of permanent continuous breach that is going on throughout the tech industry. And one of my friends said, a CISO friend of mine said to a bunch of us, is anybody else tired? And and everybody said yes. Because, effectively, we have been at DEFCON two for all of October and now going into November.
00:04:14 · Nilay Patel
[00:04:14] S2: So I'm gonna ask you some some really dumb questions here. Who is behind all these attacks? Is it Russia? Is it North Korea? Do we do we know?
00:04:23 · S3
[00:04:23] S3: It's a variety of different groups. It is it is not just because adversaries have decided, but what has happened is, from my perspective, we are at the start of the takeoff of adversary capability enabled by AI, that the open weight models have now gotten to the point at which you can automate most of the kill chain. You know? So this kill chain term is something we stole from the military. Right? In the military, they mean to actually kill something. You... There are the steps you do to blow something up. In cyber, the kill chain is the things you do to have an effect on a target. So to steal information, to turn something off, right, to steal money, to steal Bitcoin. Yeah. Traditionally, you, you know, do research upfront. You look at a piece of software. You find a vulnerability. You write exploits, and you do that all way ahead of time. And then you have a human being go and take that stuff you built and deploy it against either one target because maybe you're a government and you're going against Lockheed Martin or you're going against another government. Or you're a ransomware actor, and you take all that work you did and you deploy it against hundreds or thousands of targets really fast. Well, now with AI, what's happening is this kill chain has inverted where you can run an AI system that just goes hog wild, and it looks at the targets, and it finds the bugs in real time, and it writes the exploits in real time, and it breaks into these companies. So that's one of the things that's happening is it... These systems are going, and they're so good at finding bugs that they are finding the bugs in real time and writing the exploits in real time. And so the system that everybody's talking about is not the only one. The system we're used to talk about is a thing called RTX, which is a Chinese open source tool that anybody can go look at. There's a English translation on GitHub that you can check out. And if you power it on the back end by something like an obliterated GLM five three, it will let you break into a shocking number of organizations. This started with three South Korean banks. This week, about 200 more companies have been hit. I've been helping a couple of companies. Cognition has been kind of... You know, we're not incident response company, but we do go in and, like, help people fix their bugs afterwards by using our tools.
00:06:46 · S3, Nilay Patel
It is shocking the companies that are getting hit by something you could just download. It has been a long time since you've been able to go and just download an open source tool and break into multibillion dollar companies. It's like the nineties. Right? [00:07:02] S2: Yeah. That's something... I think I have Windows in the late nineties, early two thousands when it was a security disaster.
00:07:06 · S3
[00:07:06] S3: I started saying this actually in the beginning of this year, which is, like, it's an incredibly exciting and terrifying time because the last time it felt like this was when I was a much younger man, and I would go to the very early Defcons. You know, Defcon these days is, like, 40, 50, 60,000 people. Back in then, it was couple 100 people, and it was one one track, and you'd go in and somebody would give a talk. And you'd leave the room, and you'd be like, every company in the world is vulnerable to this attack. The difference was is the entire world didn't run on the Internet. Right? Yeah. So it's like, you could go do bad things, but you couldn't ruin the world. Right? And you certainly couldn't automate it in the way you can now. And it's back to those days where just like tools you could just go get and build yourself can go do crazy things. It's it's back to the wild west days, except we actually... Back in those days, there's, like, a couple 100 people who had those skills, and we're all in one room in Las Vegas. And most of them were actually good guys. They didn't wanna be criminals. And now you have a much larger ecosystem of people, many of whom live in places where the law enforcement can't touch and who have a big economic incentive and have access to these tools. And the other big thing is they no longer have to use frontier models that have protections, that have logs that are available to law enforcement. They can... I have a m five... My my m five Mac Ultra studio finally showed up, and I've been running, like, a quantized obliterated g l m five three. And, oh my god, like, what you can do is incredible.
00:08:47 · Nilay Patel
[00:08:47] S2: You mentioned the economics of frontier models, the... Just the economy of cyber has... Is one of those things that has gotten sophisticated over the years. Right? So you're in these early days, bunch of people in a room. One of the things that our systems got safer is you could pay the the peep... Number of people at DEF CON a whole bunch of money to protect all the systems. And it was economically smart to pay them to be defenders instead of attackers. And then I'm just rewinding my own brain through this history. There was, like, a long conversation in our industry about security through obscurity by hiding all the bugs and hiding all the code versus open source and being really transparent. There was an etiquette that developed in... About how to reveal the bugs and when to give people notification. There was transparency reports and bug bounds. Like, we built, you could call it an etiquette. You could call it an economy. We built a culture around cybersecurity and cyber defense and how the attackers and defenders would operate. What you are describing changes it all for a whole number of reasons, but the one that sticks out to me is the economics of it. Right? I can go buy a Mac studio. I can get an open weight model, and I can just start. And that does not place me inside of the culture. There's not gonna be some old man. There's, mean, Obi Wan Kenobi being like, here's how you act like a cybersecurity jet. Like, I'm just gonna start doing stuff. Tell me how that's changing right now.