Masters of Scale
00:47:00 · Neil Chilson
[00:47:00] S1: So I I I mean, I think these companies under the law are liable for the vast majority of the types of harms that you're talking about. And if those types of harms happen, they have... They will face quite strict scrutiny both from the law... The legal community as well as from their investors. And so I do think that we have incentives there to solve these problems. That's what I mean by the we. I mean that we have a structure, and maybe we could tweak it, maybe we can make it better, but we have a structure that creates an incentive to deliver products that don't harm consumers. And that that that structure that... Or that structure should benefit consumers. And I think we could tweak those incentives, maybe we could tweak the the liability, but I think that structure is ultimately the path towards the we, which is society, figuring out how to use these tools in a way that is beneficial and that outweighs the harms that they could cause. And I I think that has been the path of every single technology that we've developed in the past. It's certainly been the path of many of the technologies that we all use every day. Liability is a real thing for companies that cause harm to consumers. We need to we need to impose that. I'm not trying to let anybody off the hook, but but the but the solution is not to stop the technology and say, hey, we're not gonna improve this. It's gonna say... It's to say, hey, here are some of the challenges. Let's try to tackle these challenges. Meredith,
00:48:30 · John Donvan
[00:48:30] S2: in this best foot forward part of Neil's argument, he's making the case that the agents actually deliver a whole lot of good to individuals and empowerment and convenience, etcetera. Do you refute that piece of it? Again, setting aside the trade offs that you're talking about, do you think he's right that they have a lot of benefits to deliver?
00:48:52 · Meredith Whittaker
[00:48:52] S0: I think, you know, look, where they work as intended, which is not all the time, they are convenience tools. You know, I don't think anyone's arguing against that. And again, agents is a very woolly term. You know, for coding, agents are doing things that are very very useful. Right? Coding is a field where we have a huge amount of data and where it's sort of objectively verifiable, your code runs or it crashes your compiler and you can kind of get a sense of what's there and sort of move on. I think when we're talking about these kind of consumer agents, which is where my concern is, because, you know, they are hollowing out Signal's ability to provide privacy by demanding or coercing data access in ways that are, you know, I think architecturally irresponsible at a technical level. You know, irrespective of the convenience of, you know, not having to do your own Christmas shopping, right, you're looking at a hollowing out of our fundamental cyber security architectures in a way that is again, you know, it is reconstituting more and more power and data access in the hands of the existing winners of the AI ecosystem. So, you have agents who are sort of grabbing data from the application layer, you have agentic integrations that are becoming kind of the front end of the internet, the interface of the internet, making it harder and harder for independent website developers, independent application developers, you know, to have a place in that ecosystem. So, you know, sure they are... They can be convenient, but the argument isn't about convenience. The argument here is about should we trust them? And I think all of the evidence points to they are not trustworthy, so we should not
00:50:48 · Neil Chilson, John Donvan
[00:50:48] S1: trust them. [00:50:48] S2: The question is not are they convenient, but should we trust them? At this point, Neil, would you... You do still argue with setting aside convenience that they are trustable. I'm not saying trustworthy, [00:51:01] S1: but trustable. Trustworthy for what? I think that's the question. I think even Meredith would say signal is not appropriate to use for certain types of processes, certain types of communication. We have tools that are fit for purpose and it just depends on what the purpose is, and so I would say I trust these systems with a lot of tasks that I give them. I don't trust them with everything, I don't have them moving money around in my bank accounts or anything like that. I think they're trustworthy for the types of things that lots of people would benefit from. And I I think it's easy to dismiss convenience as not important. [00:51:42] S2: Make the case then. But convenience is important. [00:51:44] S1: Convenience aggregated across every single person who uses one of these systems is a huge benefit. It is the step forward, it is the incremental progress in technology has been a drive for convenience, a drive for making things easier. There was a time where we scrubbed our clothes on racks instead of like having washing machines. That innovation saved... It was just a mere convenience, but it saved the average family, you know, hours and hours a week of But [00:52:15] S2: did it have a downside?
00:52:15 · John Donvan, Neil Chilson
[00:52:15] S2: did it have a downside? [00:52:16] S1: Of course, it did. I'm sure the person selling the washboards was very upset of... With washing machines, and I'm sure the person selling... [00:52:23] S2: Sure. No. No. But did it have a downside as as sort of existential as what Meredith is talking about is the downside she's portraying? [00:52:29] S1: I don't think so. Well, I'm I'm not quite sure because we're we're using very fuzzy terms. I don't think that Muse is is going to, you know, kill us, and I don't think it's going to go rogue in that way. And so the question is, what are the downsides and how... So I wouldn't call it an existential risk. I think there are trade offs. There are trade offs in all these technologies. I think the vast majority of people will benefit enormously from agentic interactions even though there will be harms that are caused and there will be things that we need to solve along [00:53:01] S2: that. So you think a lot would be lost if if a wizard comes along and shuts down agentic, the agentic business?
00:53:01 · John Donvan, Neil Chilson, Meredith Whittaker, Jeff Berman
[00:53:01] S2: that. So you think a lot would be lost if if a wizard comes along and shuts down agentic, the agentic business? [00:53:08] S1: Yeah. I think basically it it... Because it's a fuzzy term, it's getting harder and harder to draw the line between what is agentic computing and what is just regular computing. There's a time at which people would have said the same thing about the internet. That that this was a this is a distributed system that nobody really controls and how are we gonna know that people are gonna use it right. And we still hear we still hear a lot about that today. Things... These technologies have benefits, they have trade offs, and and we have to work through them, and so I think this is just like those other technologies. They will have benefits, they will have trade offs. [00:53:47] S0: I I guess I don't... I mean, on the trust it for some things, not others, that's a difficult proposition for me in the context of Muse, for example, when we look at prompt injection. When we look at the sort of pervasive security concerns and the pervasive access that is required just to trust it for the most mundane tasks, it is also getting the key to the kingdom for everything else. I give you my, you know, someone the key to my house and I say, I only trust you to go in the kitchen, but ultimately, like, a couple prompt injections down the line, it is in the bedroom in the dresser drawers. [00:54:23] S3: But me