Masters of Scale
00:48:52 · Meredith Whittaker
[00:48:52] S0: I think, you know, look, where they work as intended, which is not all the time, they are convenience tools. You know, I don't think anyone's arguing against that. And again, agents is a very woolly term. You know, for coding, agents are doing things that are very very useful. Right? Coding is a field where we have a huge amount of data and where it's sort of objectively verifiable, your code runs or it crashes your compiler and you can kind of get a sense of what's there and sort of move on. I think when we're talking about these kind of consumer agents, which is where my concern is, because, you know, they are hollowing out Signal's ability to provide privacy by demanding or coercing data access in ways that are, you know, I think architecturally irresponsible at a technical level. You know, irrespective of the convenience of, you know, not having to do your own Christmas shopping, right, you're looking at a hollowing out of our fundamental cyber security architectures in a way that is again, you know, it is reconstituting more and more power and data access in the hands of the existing winners of the AI ecosystem. So, you have agents who are sort of grabbing data from the application layer, you have agentic integrations that are becoming kind of the front end of the internet, the interface of the internet, making it harder and harder for independent website developers, independent application developers, you know, to have a place in that ecosystem. So, you know, sure they are... They can be convenient, but the argument isn't about convenience. The argument here is about should we trust them? And I think all of the evidence points to they are not trustworthy, so we should not
00:50:48 · Neil Chilson, John Donvan
[00:50:48] S1: trust them. [00:50:48] S2: The question is not are they convenient, but should we trust them? At this point, Neil, would you... You do still argue with setting aside convenience that they are trustable. I'm not saying trustworthy, [00:51:01] S1: but trustable. Trustworthy for what? I think that's the question. I think even Meredith would say signal is not appropriate to use for certain types of processes, certain types of communication. We have tools that are fit for purpose and it just depends on what the purpose is, and so I would say I trust these systems with a lot of tasks that I give them. I don't trust them with everything, I don't have them moving money around in my bank accounts or anything like that. I think they're trustworthy for the types of things that lots of people would benefit from. And I I think it's easy to dismiss convenience as not important. [00:51:42] S2: Make the case then. But convenience is important. [00:51:44] S1: Convenience aggregated across every single person who uses one of these systems is a huge benefit. It is the step forward, it is the incremental progress in technology has been a drive for convenience, a drive for making things easier. There was a time where we scrubbed our clothes on racks instead of like having washing machines. That innovation saved... It was just a mere convenience, but it saved the average family, you know, hours and hours a week of But [00:52:15] S2: did it have a downside?
00:52:15 · John Donvan, Neil Chilson
[00:52:15] S2: did it have a downside? [00:52:16] S1: Of course, it did. I'm sure the person selling the washboards was very upset of... With washing machines, and I'm sure the person selling... [00:52:23] S2: Sure. No. No. But did it have a downside as as sort of existential as what Meredith is talking about is the downside she's portraying? [00:52:29] S1: I don't think so. Well, I'm I'm not quite sure because we're we're using very fuzzy terms. I don't think that Muse is is going to, you know, kill us, and I don't think it's going to go rogue in that way. And so the question is, what are the downsides and how... So I wouldn't call it an existential risk. I think there are trade offs. There are trade offs in all these technologies. I think the vast majority of people will benefit enormously from agentic interactions even though there will be harms that are caused and there will be things that we need to solve along [00:53:01] S2: that. So you think a lot would be lost if if a wizard comes along and shuts down agentic, the agentic business?
00:53:01 · John Donvan, Neil Chilson, Meredith Whittaker, Jeff Berman
[00:53:01] S2: that. So you think a lot would be lost if if a wizard comes along and shuts down agentic, the agentic business? [00:53:08] S1: Yeah. I think basically it it... Because it's a fuzzy term, it's getting harder and harder to draw the line between what is agentic computing and what is just regular computing. There's a time at which people would have said the same thing about the internet. That that this was a this is a distributed system that nobody really controls and how are we gonna know that people are gonna use it right. And we still hear we still hear a lot about that today. Things... These technologies have benefits, they have trade offs, and and we have to work through them, and so I think this is just like those other technologies. They will have benefits, they will have trade offs. [00:53:47] S0: I I guess I don't... I mean, on the trust it for some things, not others, that's a difficult proposition for me in the context of Muse, for example, when we look at prompt injection. When we look at the sort of pervasive security concerns and the pervasive access that is required just to trust it for the most mundane tasks, it is also getting the key to the kingdom for everything else. I give you my, you know, someone the key to my house and I say, I only trust you to go in the kitchen, but ultimately, like, a couple prompt injections down the line, it is in the bedroom in the dresser drawers. [00:54:23] S3: But me
00:54:23 · Jeff Berman, Meredith Whittaker
[00:54:23] S3: But me [00:54:23] S0: and this much safer than one. Is architecturally how these systems work at That's a technical not true. And so you can put harnesses in there and Muse itself is, you know, kind of... We already have the example that I shared on stage of Muse giving out a user's home address to someone else than impersonating that user in the context of a Facebook marketplace sale. Now, I'm someone who's had a stalker or two in my life. That is terrifying to me. Right? So I might trust it. Do you know, like I trust you to sell my old champagne flutes on Facebook marketplace. But suddenly that trust is breached. And there's someone at my home and Muse is saying like, I'll be right outside. I'll be right outside. I'm at the door. And somebody is very confused. Right? Who else could it give that that address to? And so I don't... I I just... I think we need to recognize that again, at the level of the fundamental architectures and the the way these agents are being integrated at the consumer level, it's very difficult to just sort of cabin off. I trust you just to do this task. And in fact, a thing that did one task and checks in every time is not an agent in the current collusional It's
00:55:36 · Neil Chilson, Meredith Whittaker
[00:55:36] S1: it's just maybe... It's it's an agent that's learning how to be trustworthy. And I think the Muse example is is particularly interesting because there are plenty of things that I would not advise people to install, say, OpenClaw on their home computer because it's their home computer and their computer has lots of things. But with Muse, we have an isolated computer that is in Meta's cloud, and that means that if something goes wrong on that computer, it doesn't mess all the things up that are on my computer. That is safer for consumers in many ways than installing an agent on their own device. And so in that way, there's a benefit. And then the example about the the Facebook marketplace, I am not sure how people deliver champagne flutes to somebody who bought them without arranging a place for them to meet, and so some of this is the the user learning what it means to charge an agent with a task. This agent did not hand out the address to random people, it handed out the address to the person who purchased something. [00:56:42] S0: Who wanted to purchase it. [00:56:44] S1: Who wanted to purchase it. And that's... [00:56:45] S0: That the user who handed it... Muse the task had no idea this is how it was working. [00:56:51] S1: I I I... Like I said, things will go wrong and people will learn. I think...