Decoder with Nilay Patel
00:08:47 · Nilay Patel
[00:08:47] S2: You mentioned the economics of frontier models, the... Just the economy of cyber has... Is one of those things that has gotten sophisticated over the years. Right? So you're in these early days, bunch of people in a room. One of the things that our systems got safer is you could pay the the peep... Number of people at DEF CON a whole bunch of money to protect all the systems. And it was economically smart to pay them to be defenders instead of attackers. And then I'm just rewinding my own brain through this history. There was, like, a long conversation in our industry about security through obscurity by hiding all the bugs and hiding all the code versus open source and being really transparent. There was an etiquette that developed in... About how to reveal the bugs and when to give people notification. There was transparency reports and bug bounds. Like, we built, you could call it an etiquette. You could call it an economy. We built a culture around cybersecurity and cyber defense and how the attackers and defenders would operate. What you are describing changes it all for a whole number of reasons, but the one that sticks out to me is the economics of it. Right? I can go buy a Mac studio. I can get an open weight model, and I can just start. And that does not place me inside of the culture. There's not gonna be some old man. There's, mean, Obi Wan Kenobi being like, here's how you act like a cybersecurity jet. Like, I'm just gonna start doing stuff. Tell me how that's changing right now.
00:10:09 · S3
[00:10:09] S3: Effectively, the last era of security from my perspective started with the Microsoft... The Bill Gates trustworthy computing memo in the early two thousand, and that is when we started designing how we do software security. And these big companies, Microsoft and, you know, Oracle, and then later Amazon and Salesforce, and these folks started hiring ex hackers. And I was part of that generation of the ex hackers who were, like, at DefCon and Black Hat, got hired by the companies to go break their stuff. And we started coming up with responsible disclosure, bug bounties, consulting companies. So the Foundstone and At Stake and then Mandiant and CrowdStrike. These companies grow up, and we get this huge industry that is now RSA and Black Hat, and these... This multibillion dollar security industry grew up in the two thousands and twenty tens. That era is over, like you said, because it it was based upon a couple of assumptions. One was that there was a hierarchy of skill sets of people who could do this stuff, that you had the script kiddies who could download tools to do things, and they're then the next level up of the people who could write those tools, and that that was more... A more verified skill set. And then there was the kind of vulnerability researchers, and then there was kind of the gods. And that god tier was in the hundreds of people, and that is the the people who do the really, really good vulnerability research, who could write the really, really good shell code, who could reverse engineer patches from a Microsoft or or or somebody like that. Also, assumption that there's a limited number of people who could be operational in an attack. And so if you wanted to build an attack team that you have a small r and d component, and then you have a much larger operational component. And so if you are a professional attack team at US Cyber Command, at Russia SVR, at Chinese MSS, that you you build the structure of your r and d team of elite people, and then your less elite operational people. And then that was copied then by the, like, the Russian ransomware actors, and that you have to have a conspiracy of people to kinda do this stuff.
00:12:18 · S3
A lot of this stuff also fell apart with the invention of Bitcoin because one of the assumptions was that you had to have people who could launder money, and the Bitcoin meant that nerds could launder money. So that that started to change the economics Yeah. Of the the entire thing. All this stuff is falling apart because of AI, because the skill set issue is that now script kiddies have a bunch of the skill set of the Dave Aitel and the other cyber gods who could do these things, that with this Mac that's sitting on my desktop, I can reverse engineer a Microsoft patch and walk through and then write stable exploit code. It's not quite there yet to be automatic. You still have to have a bunch of skills, but we're really, really close. And then what's really concerning and what you see with the RTX stuff is you can now kind of let loose on the operational side. We were never secure. It was all like a fantasy. Right? These companies... These vulnerabilities have always existed. These these companies have never been safe, and and that has... Is what we've seen over the last decade is these mostly Russian ransomware actors nailing kind of the midsized companies that most people have never heard of but are actually really important, like some midsized industrial company that makes some widget that we all depend upon. I once did work for, like, the largest distributor of of medical and dental supplies in the country, and they got really hit badly by a Russian ransomware actor, and then, like, dental offices across the country were, you know, canceling appointments. Didn't even make the news. Right? But because they couldn't get gloves, because they couldn't get supplies. And it's because, you know, a company like that, it's got, like, one security guy and a 10 person, 12 person IT team, and they're going up against a 20 person Russian ransomware team. They're toast. Right? But it's just because there's only a handful of Russian teams that could do that. Now two or three Russian kids can have a whole bank of Macs, or they have the money. They can buy NVIDIA h one hundreds, and they can run a ton of agents who do this all day, and that is what is changing. So I'm sorry. You asked who was doing this. For Binance, it's the North Koreans, and they have an entire team that all they do is steal Bitcoin.
00:14:35 · S3, Nilay Patel
Other people have gone out of it because it's, like, really hard to compete against a foreign intelligence agency that recruits their smartest kids, and all they do is steal Bitcoin. They're extremely good at it. And for the Binance hack, we could talk about it. There's some... Probably some AI component, but mostly that was actually human intelligence and very, very smart social engineering. And the North Koreans are... I mean, you have to give it to them. They're very, very good at that. Yeah. But for the other stuff, it's it's... It looks like for the the people who are breaking into the AI companies is what's called the Chinese cottage access industry, which are small Chinese contractors who break into companies and then sell the information to much larger Chinese companies. [00:15:22] S2: That diffusion of skill changing the sort of economics of cyber is really fascinating to me. We ran a headline ages ago called Attack of the Killer script kiddies at, like, the very beginning of all this. And I feel bad because it is much more suited to now than it was to then. Because then it was just a lot of, like, automated mischief. [00:15:40] S3: Because you wasted the headline? You you wish you held it back?
00:15:40 · S3, Nilay Patel
[00:15:40] S3: Because you wasted the headline? You you wish you held it back? [00:15:43] S2: I I... You know, every now and then, you get a lightning bolt of headline. You gotta you gotta get it out there. It's a great piece of art. You know, we'll we'll put it in the show notes. But it feels like there... The diffusion you're talking about is not, oh, a bunch of teenagers are gonna be even more malicious. It's a bunch of bad actors with real incentives to do harm are now vastly more capable than they were before. And that doesn't seem like you can rejigger the economics of that unless you somehow make it really expensive to do the attacks.
00:16:16 · S3, Nilay Patel
[00:16:16] S3: I think it's gonna be extremely hard to make it expensive to do the attacks. We're gonna have to lower the cost of defense. And you're completely right to look at this economic problem because what's happening is attacks are becoming agentic. The cost per token for attackers is going towards zero, and the cost for defenders is staying quite high. And so it is very important for us to find ways for defenders to have cheaper options, and I think that is critical. So that's something I am personally extremely focused on, and I think, to be frank, very few people are. There's a couple of problems here. One, there's a ton of focus on the defensive side of AI on finding and fixing bugs. That's great. It's important. But software security as a CISO, like, I work for an AI software company, Cognition. Even as a CISO of a software product company, software security is 25% of my job. Right? I have all kinds of other responsibilities, and software security is the part of my job that has been the most both revolutionized and improved by AI. The other 75% has been helped a bit, and in some cases made much worse. So there's so much more we need to do than just hill climb on finding and fixing bugs. There's been so much less hill climbing on all these other things, and the next area needs to be the operational security side. [00:17:58] S2: Give me a... Wait. Give me an example of the other things, the 75.