Decoder with Nilay Patel
00:46:54 · Nilay Patel, S3
[00:46:54] S2: This is, if you add it all up, this is a good argument to slow down. Right? It... To say, we need to catch up to the capabilities, the models. We need to build some of these systems. We need to initiate everyone into a way of thinking where the cost of prevention is is worth it and not just the cost of the cure. Do you think that it's likely that we we will slow down? Do think that we should? [00:47:16] S3: If I had my druthers, I would be fine slowing down on the frontier and then catching up on the cheap... Cheaper models for defense. I'm a defense guy. I have to, like, live in the world that other people who work six blocks for me Yeah. And, you know, who are way, way, way, way richer. Make... I live in their world, man. Yeah. So, like, I get asked this question all the time, and it's like, I just do the best I can within the world other people make. [00:47:44] S2: Yeah. Do you think that the the big labs have woken up? You you mentioned that their cybersecurity teams, they've gotten more access. They've they've stopped what you might call the hugging face error, or at least they've they've changed the structures of those companies. Do you think that will help?
00:47:58 · S3
[00:47:58] S3: I mean, certainly, they've woken up on the eval risk. Right? On... I think they have teams who who have been the... This entire time working very hard on the big picture cyber risk. There's a bunch of people in these in these companies, as you know, who want to slow down. Yeah. A lot of that discussion is driven by the p doomers who... It's about AI killing us, and I have never... I've... That's never been my that's never been my concern. Right? And to me, that has always been a driven by 24 year olds who all they've done is AI, and they've never done anything else. Right? Like, you and I are both, I think, old enough to remember. I grew up a couple miles from a sack base, and we did drills where I was taught that hiding be... Behind beneath my public school desk would save me from a Soviet thermonuclear bomb. Right? Like, that was p doom. You know? Mhmm. My grandfather built a bomb shelter that my mom was taught to hide in during the Cuban missile crisis. That was p doom. And so, like, there's, like, a a youth issue and a never doing anything but AI issue of of of people who, like, they got their undergraduate, their master's, and their PhD only doing linear algebra and and neural networks and never touching anything else. And so I have always been much more concerned about the human uplift risk from AI, of all the risks about human beings using AI to do bad things to other human beings. Right? And and so part of my concern is that there has been success in these labs working together on, like, we're not gonna let them encrypt their chain of thought. Because when they talk about pacing the frontier, it's for the p doom stuff, for the the polycules that live together indirectly. You know, like, they're talking about very specific stuff about keeping control of the models themselves, which I think is great. I think that stuff needs to happen. But then if those things happen, then there's still often an increase in a bunch of capabilities that human beings can use. And and so I'd really prefer there to be kind of a broader discussion on, could we make these things, one, less human seeming? I'm a big fan of, like, I don't think these things should have personality. I think the goal of these things should be to serve humans. They should not seem to have souls. They should not seem to have personality. I think a lot of the negative impacts...
00:50:28 · S3, Nilay Patel
As somebody who was at Facebook and saw what it was like for that company to have negative impacts on people, I think there should be a much broader feeling at these companies of, like, what are we doing kinda broadly to people? And a lot of that could just be, don't have these things ever act like somebody's friend. [00:50:49] S2: Yeah. [00:50:49] S3: Right? At a maximum, it should act like... Like, the people... I I don't know what the company is, but the people who made the, hi. I'm, a really nice... You know, like, the photorealistic Yeah. Girl who will chat with you, that is, to me, spectacularly unethical Yeah. To even try that, to even start that. That's the kind of thing that we should just make... I don't know if we can make it illegal, but we should make it completely societally not acceptable to work on that kind of stuff.
00:51:18 · Nilay Patel
[00:51:18] S2: That's a great segue because I I did wanna ask you about Facebook and content moderation. Like I said, you and I first started talking a long time ago about content moderation in the context of security. Right? You would get foreign nation states flooding our algorithms with all kinds of misinformation during elections, and that was a security problem. I I opened Instagram today, and I am confronted with some of the most nightmare copyright law hypotheticals to have ever existed in the history of the world. Right? It's a... The one I got today was a deep fake of Lewis Hamilton and Charles Leclerc doing somebody that I used to know. And I was like, well, like 50 different people can probably sue Meta right now. Right? And also, this is hilarious if you know what happened in that last race. So, you know, we keep talking about it. We've posted some clips of us talking about it on The Vergecast where I'm like, I I think Meta is letting their AI ambitions crush Instagram. Right? Like, is being flooded with slop in a very particular way. Some people love it. Sometimes it's funny. But there's a nightmare set of lawsuits coming. There's a whole rights management problem coming, and then the users don't like it. And they should label it and let you filter it. And so we post this, and Adam is serious. He said this to us in a reply on threads. I would love to do this. I'm worried that the models are gonna get ahead of us. Right? That we can try to do this, but there's not actually a way to recognize AI content. There's not actually a way to label it well. If you'd... Or if you're worried about the sexy AI influencers stealing jokes from a comedian, Now you gotta build a database of jokes. You have some history here. I I think Instagram is about to go through it in a real way. Is there some technical solution to the problem of AI slop in social media?
00:53:02 · S3, Nilay Patel
[00:53:02] S3: I think he's right that it's not perfect, but also something I noticed during my time at Facebook was when Facebook doesn't want to do something, they will allow the technical challenges to stop them. Yeah. When Facebook really wants to do something, they will give it their all. Right? And that is the pattern always. So when I wanted to try really hard on the safety issue, man, this is not solvable. And you're right. It's never perfectly solvable, but let's give it a shot. Right? But, like, whenever Zuck is told something is impossible, it's impossible to monetize mobile. It's impossible to, you know, build this network to 2,000,000,000 people. It's gonna be, you know, impossible to monetize WhatsApp. It's gonna be impossible to work AI into the system while maintaining end to end encryption. He's been told that everything's impossible over and over again, and then he constantly succeeds. They will give it a shot, and they will do it. And... But when they don't wanna do something, oh, that's impossible. So... [00:54:07] S2: Yeah. Do do you think that there's some way to understand that some piece of media is generated by AI in a consistent and verifiable way? This to me seems like the challenge.
00:54:18 · S3
[00:54:18] S3: I am not as up on the literature here. I think up to this point, there has been a pretty decent arms race. By definition, AI will continue to get better because the way... Especially for visual models, for diffusion models, the way they work is that you have detection oracles that they test against. Certainly, you can never have something open source because it will immediately end up in the diffusion model. Right? But for closed source Oracles, where then you do not make it easy for them to test the Oracle, one of the key things you do in trust and safety is then you have delayed enforcement mechanisms, so they can't brute force and and test. I think it is quite possible. Let's say I was still at Meta. My advice to them would be to then also explore a variety of of different detection mechanisms outside of the content itself. Right? And so I would do the best I can on the content, but I would also be doing low level access as possible to the camera. I would only let content to come from the camera. And what I would say is, Nilay's completely correct, there's a massive hunger for, especially among young people, stuff that is authentic. I have three teenagers, and they hate AI. They hate AI slop. They have a massive negative reaction. My wife and I created just, a cute little family AI thing, and they, I mean, they yell at us about a bunch of stuff, but they lost their minds about this, like, cute little family AI thing. Right? Be... They just hate it. And so I think there's... Somebody is gonna come and disrupt Instagram, and it's gonna be because... There's already... I've seen these little things where it's like you take the photo during the day, like, real and stuff. There's like these little things, but somebody's gonna massively disrupt Instagram because they're taking the images of the photos straight off the camera, and then all the editing is going to be only within the app, and they are going to do things to make sure that nothing is ever... Nothing can make it into the pipeline from AI, and that you're going to allow some kind of cap cut like functionality that never has an AI component. And I think that is going to be a killer app.