Masters of Scale
00:00:39 · John Donvan
[00:00:39] S2: The promise of artificial intelligence agents and assistants is that they can go out into the world and complete tasks and check things off our to do lists. They can book our travel, they can make dinner reservations, they can schedule meetings. But is handing over our credit cards and our passwords and basically our whole life to an anonymous autonomous agent too risky? Welcome to Open to Debate. I'm John Donvan, and today a debate around those risks. We did this one on stage at the Masters of Scale Summit. That's an annual event held in San Francisco that brings together business leaders. They were our partner for this episode and the question that was debated: Should we stop trusting AI agents with our information? Our debaters included somebody who has long been a voice on the issue of privacy that's Meredith Whittaker of Signal she argued not to trust and challenging her, Chilson. He's a lawyer, a computer scientist, and author who heads up AI policy at the Abundance Institute. And we had a guest moderator for this one, Jeff Berman, a former public defender turned entrepreneur who heads up the media company that produces the summit. And after the debate, Meredith and Neil will join me in the podcast studio for a debrief.
00:01:46 · Jeff Berman
[00:01:46] S3: With us today, Meredith Whittaker, the president of Signal, and Neil Chilson, the head of AI policy at the Abundance Institute. We have a tight clock. I'll remind you that the resolution on the table is, should we stop trusting AI agents with our information? Three minute opening statements for each of you. Meredith, we'll start with you.
00:02:05 · Meredith Whittaker
[00:02:05] S0: Alright. I don't know. Did we start trusting agents with our information or are we just kinda using them because we need to? Right? Look, I run Signal, and my job is pretty serious. I need to protect people's data because they use Signal for privacy in contexts where it's often a life or death situation. And so this is not a theoretical debate. I don't really care about winning. What I care about is facing these issues clearly and coming up with solutions so we can build trustworthy technology that actually deserves our trust. Now, look, you want to use an agent, it is convenient, you want to book your birthday party and invite your friends. That's a great agentic marketing case, and it's the kind of thing we hear from agent companies all the time. But, of course, in order for an agent that is running on your consumer device to do that, it needs access to your contacts, it needs access to know who your friends are, it needs your credit card information, it needs to be able to drive your browser and effectively impersonate you to search for a restaurant, to put your credit card down, to make that reservation. It needs, in this case, maybe access to your calendar to figure out what dates are available for you and your friends. And then if you're using Signal, like you probably should be, it needs access to your Signal messages to communicate with your group text and to figure out with your friends what actually works to book that party. So, on the one hand, you have the promise of a magic genie that's gonna, you know, do something that's really convenient. Right? This stuff takes time. It can be inefficient. And on the other hand, you have the paradigm of what a consumer agent is doing right now. Which is, one, it needs deep permissions, what we'd call root permissions in, you know, my technical world. It needs to be able to act without explicit consent, doing a number of steps to achieve a goal, or at least it thinks achieve a goal. And it needs to be able to respond to natural language, to text, in context, as if that text were a command, and follow that text command. And this is where we get to prompt injection, which is an extraordinarily serious issue, and this is how you hijack an agent. There's already a huge number of examples of prompt injection leading to problems. A zero click exploit pulled corporate data off of Microsoft's Copilot.
00:04:41 · Meredith Whittaker, Jeff Berman
You have Perplexity's agentic browser hacked by a Reddit comment via prompt injection. You have 11% of Anthropics Claude Copilot experiment, or Claude Chrome experiment, 11% of the time, even with guardrails, it was not able to resist prompt injection. We have researchers showing a prompt injection attack on Google Calendar that led an agent to turn on the boiler in someone's smart home. So when we're talking about trust, I think the answer is fairly obvious. No, of course you shouldn't trust these fallible systems, But we should be pushing to build trustworthy systems. Because in security, trust comes with verification. I'm not asking you to signal. I'm not asking you to trust me. I'm asking you to look at our code and recognize that it's trustworthy, and if it isn't, come back to me, and we will fix it. [00:05:34] S3: Thank you. Neil, three minutes and twelve seconds. Great. So
00:05:42 · Neil Chilson
[00:05:42] S1: for twenty years, the Internet has been built on a bargain. Data that... About you that you couldn't use, companies pooled, they sold ads against it, and they used that money to fund very useful services. That bargain, meant that companies learned a lot about us, but we didn't learn that much about ourselves. AgenTic AI flips that bargain. We all now have, each of us has access to powerful software tools that can comb through the mountain of our data for us. So, I use a lot of the applications, a lot of the agent use cases that Meredith referred to. I give it access to my calendar, it manages that, it monitors my family's finances, I use it to troubleshoot all the time. Thousands of people do lots of other things that are like that, you know, comparison shopping online, and I I take the point that a lot of that's driven by convenience. But these are tasks that only the wealthy could delegate in the past. If you don't have an executive agent, these are the types of things that you have to do yourself. So, there is a huge amount of convenience here. Agents give us agency, the theme of this event. And yes, that does mean that we have to share our data, but that data is being used for our purposes, not the old bargain, of the Internet. And here's the surprise. These agents can... One of the things that they can make more convenient are some of the practices that we might use to increase privacy. So today, we trade privacy for convenience all of the time. But agents don't care about privacy. Or... Sorry. They may not care about privacy. We'll find out. But they they don't care about inconvenience. They don't care about inconvenience. An agent doesn't get bored reading privacy policies or debating between them. It doesn't get bored... It doesn't get confused by using Tor or NoScript in order to, you know, get through some of these complicated things. These are tools that humans find very burdensome and cumbersome. Agents don't care about that. They can do those types of practices for us. And so agents have the time and the patience to make privacy more convenient for us. Earlier today in the, in my my hotel room, I used Muse to unregister me to block, block me on five of the major, data brokers in The US, and and it took a six word prompt. That was helpful.
00:08:16 · Neil Chilson, John Donvan, Jeff Berman
Now, has spent her career protecting the privacy of communications, and so obviously, she's reasonably worried that agents will open a new window into our private lives. But look at the status quo. There are many windows into our private lives and I hope to convince you that agents can stand on our side of the glass. They can help us track the trackers and they can help us close the windows that we never even intended to open. So I totally agree with Meredith, we need to build more secure systems, but agents can improve the old internet bargain. They can do this by making privacy easier, and more convenient, and they can do it by having you use your data rather than somebody else. [00:09:03] S2: Thank [00:09:03] S3: you, Neil. Meredith, I hear Neil saying that the agents give us an opportunity not only for a better privacy bargain, but we can actually enhance our data privacy by using the agents. Why is he wrong?