Masters of Scale
00:18:50 · Meredith Whittaker
[00:18:50] S0: Well, I think there are people trying to do that. But, again, we need to look at what is the promise, the marketing promise, that is being sold to people around these consumer agents. And it is something similar to having a magical genie servant that can do everything for you. Now, in order to do everything for you, it needs to know everything about you. And, yes, you can put harnesses on, you can add more sandboxes, you can have a sort of threshold that if it accesses this or that database, or is about to take an action, it has to check-in with me. But an agent that checks in at every step is kind of not an agent. An agent with no access to data doesn't quite work to fulfill that promise, and an agent that doesn't have access to all of your applications isn't really able to do that. So there is an issue here, paradigmatically, with what an agent is and what it does that I think gets in the way of some of that. And we... You know, again, cybersecurity could not be more prominent right now. I've been talking about this for about two years. We now have sort of, you know, the proof points are blowing up everywhere. And I do think we need to walk back some of this hype, some of this this marketing, and just get back down to technical fundamentals. Right? If an agent is screenshotting everything on your device, keeping that in a database, processing that off-site, you have just broken Signal's security model. You are eliminating the ability for privacy at the application layer. And, you know, I think in in contradiction to your statement agents give us agency, I see this move as pulling agency back to the AI companies, to the big OS vendors, as they grab data from independent developers, from websites, as they're shutting down the APIs at the back end, this is a data grab. And Amazon's lawsuit against Perplexity's agent, because Perplexity is in their marketplace, gathering a huge amount of data that Amazon does not want to share with a third party shows that this is actually a struggle between companies for data very much in line with the traditional Internet business model.
00:20:56 · Jeff Berman, Neil Chilson, Meredith Whittaker, John Donvan
[00:20:56] S3: Is there is there such a thing as a safe agent? [00:21:01] S1: Is there is there such a thing as a safe agent? Not in not in human agent or in computer agents. [00:21:09] S0: Okay. And there [00:21:10] S1: are isn't such a thing as safe computers. That's true. I don't... That's true. But but... Would you trust that unsafe or what what is the question? Is the real question. And so when you talk about having to give entire access, root access to your agent, that's not true. You can choose what access you give these agents and they're built to tier it this way, And yes, when you get a lot of prompts back, you can decide whether or not the agent is acting in a proper way and the agent can be trained to be better at those tasks. And yes, we should take small steps, trusting steps, we should test, but we should verify, and that is true, and I think that's what users should do. That should be the advice. The advice is don't interact with these things. The advice is figure out how you can use it safely. Try it out and let's see if we can make them better, and you don't have to go a whole hog and try it on everything. I do. I certainly don't. And so I think that this all or nothing idea around agents just doesn't work. It's first of all, it's just not going to happen. We're people are going to build these agents and if people aren't building them with privacy in mind, then the people who are building them are gonna be people who don't have privacy in mind. And I think that's a worse... [00:22:18] S2: So, Meredith,
00:22:18 · John Donvan, Jeff Berman, Meredith Whittaker
[00:22:18] S2: So, Meredith, [00:22:19] S3: I I subscribe to a lot of newsletters. Why why shouldn't I just start with saying, well, I'd love for you just to read this email address where I've cabined it to my newsletters and give me a summary and, like, can we take baby steps toward agents until we get to a place where there is such a thing as safe agent? [00:22:34] S0: Yeah, I just think we need to define the pronoun we. Like, were you aware that Meta was grabbing your contact list? That Meta was creating dossiers of everyone you talked about, everyone you spoke to, you know, in email and messaging, when you installed Muse? No, I'm probably not, because I wasn't aware, because it took investigative journalists digging into this, and Meta dissembling a little bit to get this information out in the public, because it wasn't on the front page of what Meta was saying Muse did. Because we are habituated to opaque software hidden behind secretly, and a lack of technical understanding by the majority of people who don't have a narrow education in CS to say, hey, this looks convenient, it seems like everyone is doing it, click through the TOS click wrap, and then you have a thing installed on your your phone that looks a lot more like malware than it does like traditional software. [00:23:31] S3: We we are out of time, but, Neil, before we go to closing statements, just wanna give you a chance to respond to that if you'd like.
00:23:31 · Jeff Berman, Neil Chilson
[00:23:31] S3: We we are out of time, but, Neil, before we go to closing statements, just wanna give you a chance to respond to that if you'd like. [00:23:36] S1: I mean I mean, that's the exact problem that I think agents could help with and having an agent that could look and read that click wrap and surface those types of issues very quickly is very useful. That is very useful to the average person. Now I happen to have a master's in computer science, but I know most, many people don't, and so, and so I was aware of the the issues and the data that Muse was going to use, and I thought it was a good trade off, and I accept that many of you don't. And so I do agree with you. The pronoun on we does matter. I think that agency is about letting people choose in this space, and giving them tools that help them choose better in more informed ways. I think agents can help with that. [00:24:13] S3: Thank you. We'll move to closing statements. Two minutes for each of you, Meredith. We'll start with you.
00:24:17 · Meredith Whittaker
[00:24:17] S0: Well, look, again, I think it's really important that we come down to technical brass tacks here, and recognize that there is a real issue with these systems. Agency is about choice, but if you choose for me, if you choose to install Muse, and Muse is sucking up my communication data and our data, that's not really a choice that I made. And I think it is a mistake to look at this as sort of an issue where individuals are just choosing to use this, given the vast distinction between the marketing that people are hearing about super intelligent, conscious AI that can do everything magically on the one hand, and the technical reality of what it takes to make these real life systems that depend on the hardware and the software and the libraries and the supply chains and the device access of agents actually function to do these things. And I would, you know, again say that what we are looking at as Signal is extremely stark. We're looking at the introduction of agents into the operating system in ways that are nullifying or threatening to nullify our ability as independent application developers to protect your privacy and security at the application level. Because when you are using the accessibility tool kit, when you are poisoning the water, the operating system that Signal and all other applications swim in, in order to grab data, in order to execute an agentic command, our encryption, which has been verified, which is formally verified, which is sort of mathematically proven to be robust, that protects your privacy, that means you don't have to trust me, you don't have to trust Signal Based on Vibes, You can look at the code, and you can verify that trust is earned. That's no longer the shield. The shield is whatever that agent that now has access to your messages may or may not do with it. That is the vector of attack, and that is hollowing out the idea of privacy and security at the application layer, at the website layer. It is actually reversing the promise of an open Internet where everyone can participate and grabbing data and authority back to the massive companies already owning AI.
00:26:30 · Jeff Berman
[00:26:30] S3: Thank you, Meredith. Neil, two minutes for your closing.