AI broke the cybersecurity world. Now what?

00:51:18 · Nilay PatelPlay from here

[00:51:18] S2: That's a great segue because I I did wanna ask you about Facebook and content moderation. Like I said, you and I first started talking a long time ago about content moderation in the context of security. Right? You would get foreign nation states flooding our algorithms with all kinds of misinformation during elections, and that was a security problem. I I opened Instagram today, and I am confronted with some of the most nightmare copyright law hypotheticals to have ever existed in the history of the world. Right? It's a... The one I got today was a deep fake of Lewis Hamilton and Charles Leclerc doing somebody that I used to know. And I was like, well, like 50 different people can probably sue Meta right now. Right? And also, this is hilarious if you know what happened in that last race. So, you know, we keep talking about it. We've posted some clips of us talking about it on The Vergecast where I'm like, I I think Meta is letting their AI ambitions crush Instagram. Right? Like, is being flooded with slop in a very particular way. Some people love it. Sometimes it's funny. But there's a nightmare set of lawsuits coming. There's a whole rights management problem coming, and then the users don't like it. And they should label it and let you filter it. And so we post this, and Adam is serious. He said this to us in a reply on threads. I would love to do this. I'm worried that the models are gonna get ahead of us. Right? That we can try to do this, but there's not actually a way to recognize AI content. There's not actually a way to label it well. If you'd... Or if you're worried about the sexy AI influencers stealing jokes from a comedian, Now you gotta build a database of jokes. You have some history here. I I think Instagram is about to go through it in a real way. Is there some technical solution to the problem of AI slop in social media?

00:53:02 · S3, Nilay PatelPlay from here

[00:53:02] S3: I think he's right that it's not perfect, but also something I noticed during my time at Facebook was when Facebook doesn't want to do something, they will allow the technical challenges to stop them. Yeah. When Facebook really wants to do something, they will give it their all. Right? And that is the pattern always. So when I wanted to try really hard on the safety issue, man, this is not solvable. And you're right. It's never perfectly solvable, but let's give it a shot. Right? But, like, whenever Zuck is told something is impossible, it's impossible to monetize mobile. It's impossible to, you know, build this network to 2,000,000,000 people. It's gonna be, you know, impossible to monetize WhatsApp. It's gonna be impossible to work AI into the system while maintaining end to end encryption. He's been told that everything's impossible over and over again, and then he constantly succeeds. They will give it a shot, and they will do it. And... But when they don't wanna do something, oh, that's impossible. So... [00:54:07] S2: Yeah. Do do you think that there's some way to understand that some piece of media is generated by AI in a consistent and verifiable way? This to me seems like the challenge.

00:54:18 · S3Play from here

[00:54:18] S3: I am not as up on the literature here. I think up to this point, there has been a pretty decent arms race. By definition, AI will continue to get better because the way... Especially for visual models, for diffusion models, the way they work is that you have detection oracles that they test against. Certainly, you can never have something open source because it will immediately end up in the diffusion model. Right? But for closed source Oracles, where then you do not make it easy for them to test the Oracle, one of the key things you do in trust and safety is then you have delayed enforcement mechanisms, so they can't brute force and and test. I think it is quite possible. Let's say I was still at Meta. My advice to them would be to then also explore a variety of of different detection mechanisms outside of the content itself. Right? And so I would do the best I can on the content, but I would also be doing low level access as possible to the camera. I would only let content to come from the camera. And what I would say is, Nilay's completely correct, there's a massive hunger for, especially among young people, stuff that is authentic. I have three teenagers, and they hate AI. They hate AI slop. They have a massive negative reaction. My wife and I created just, a cute little family AI thing, and they, I mean, they yell at us about a bunch of stuff, but they lost their minds about this, like, cute little family AI thing. Right? Be... They just hate it. And so I think there's... Somebody is gonna come and disrupt Instagram, and it's gonna be because... There's already... I've seen these little things where it's like you take the photo during the day, like, real and stuff. There's like these little things, but somebody's gonna massively disrupt Instagram because they're taking the images of the photos straight off the camera, and then all the editing is going to be only within the app, and they are going to do things to make sure that nothing is ever... Nothing can make it into the pipeline from AI, and that you're going to allow some kind of cap cut like functionality that never has an AI component. And I think that is going to be a killer app.

00:56:29 · Nilay Patel, S3Play from here

[00:56:29] S2: Man, I could do another whole hour with you on this because the the the system you're describing there is not detecting AI content. It is verifying real content. [00:56:38] S3: It's much easier. Verifying real content is much easier. Right? And and there's been a big focus on this on the professional side. Right? Like, Adobe has a whole thing of, like, if you're a New York Times photographer, you have a $5,000 Nikon or Canon. You take the photo, [00:56:51] S2: and it's here has that in it now. [00:56:54] S3: Right? Like, there's a lot of this now. Yeah. [00:56:56] S2: There's a there's a lot of this out there. It's... But it's the other side that I'm I'm thinking about. Right? Where it's... We are going to detect AI text. Right? The the the leading model companies, because of European regulations, are now, like, futzing with the text they generate so they [00:57:10] S3: Text is much text is much harder because you just have so much less data. Right? Like, especially if a 140 characters, like, what do you have in there to detect? You know, the more you've got, the more you can do. [00:57:19] S2: It is amazing because I can detect Claude a mile away. So may... Maybe maybe I'll just rely on my own [00:57:24] S3: human judgment. Not in a... Not in tweet length. [00:57:26] S2: That's true. The the cheap ones, can. But I'm just saying, like, the notion that we're gonna verify what's real instead of detecting what's fake feels very important in the context of the media we consume.

00:57:26 · Nilay Patel, S3Play from here

[00:57:26] S2: That's true. The the cheap ones, can. But I'm just saying, like, the notion that we're gonna verify what's real instead of detecting what's fake feels very important in the context of the media we consume. [00:57:40] S3: I think detecting what's fake is always gonna be probabilistic, and we're gonna asymptotically approach impossibility, whereas detecting what's real can be cryptographically perfect. So I I would not pick the battle that we will eventually lose. I would pick the battle that we can cryptographically win. That would be my suggestion. [00:57:58] S2: And there's somewhere in there where you have to build an entire chain of custody. Right? We're gonna take the sensor data off the camera. We're gonna put it in a cloud server. We're only gonna allow edits on this server. That file will never get monkeyed with. [00:58:10] S3: Sure. That's true for the New York Times. [00:58:13] S2: Yeah. [00:58:13] S3: And you know, Adobe Photoshop and stuff, but not for Instagram. For Instagram, you can do it all because it's it's only the iPhone, and it's only Instagram. So, like, you... The entire chain of custody is within your app. It is within the iPhone sensor. It's in Instagram. So, like, it's actually easy for them. [00:58:29] S2: Look. Like I said, we could do another whole hour on that because it feels like right next to the cybersecurity problem is the social engineering problem, and right next to the social engineering problem is the fake media problem. Right? Like, how many phishing attacks are just this website looked pretty real? And

00:58:29 · Nilay Patel, S3Play from here

[00:58:29] S2: Look. Like I said, we could do another whole hour on that because it feels like right next to the cybersecurity problem is the social engineering problem, and right next to the social engineering problem is the fake media problem. Right? Like, how many phishing attacks are just this website looked pretty real? And [00:58:46] S3: Oh, yeah. [00:58:46] S2: Or this person is talking to you, or there's an AI voice on the other side of the phone. And all of that comes down to, can you detect what's fake instead of verifying what's real? And I Oh, no. [00:58:55] S3: I I've had to train my... Yeah. I've had to train my parents that if you get a video from your grandchildren saying they've been kidnapped, you have to call me because the odds that it's real is is not very high. [00:59:07] S2: Yeah. [00:59:07] S3: Right? Like, that's where we're at as a as a society. [00:59:10] S2: I feel like I wanna end on a hopeful note, and I don't know how to do it from there. [00:59:14] S3: Wrong guess for that, buddy. Yeah. [00:59:17] S2: This is why you only come out every few years. But now after this, we're gonna we're gonna do the series of, like, birthday party guests. So you... You've been talking about the dark ages. You you... We're in it, right, where it's starting to accelerate. What are the markers people should look for in terms of progress? Right? To say, okay, we're climbing our way back up.