Masters of Scale
00:14:01 · Jeff Berman, Meredith Whittaker, Neil Chilson
[00:14:01] S3: point about us being privileged people, I'm guessing you have an executive administrative assistant, many of them in the room do. They probably have access to an enormous amount of your life, your contact information, etcetera. Most people can't afford to have someone do that. And this is a massive time saver, a massive efficiency creator for people who don't have the means otherwise. What's really the difference between you're having a human admin and you're having a digital admin? [00:14:27] S0: Well, Emma, my assistant, is incredible, but if Emma messed up like this, I would fire her. There is labor contract that protects my relationship with Emma. She understands, as a human being, how sacrosanct the information that she has access to is, and she's accountable for it. She's not a bot that is installed at a pervasive level, who's sort of fumbling through text, you know, that that could lead to a serious catastrophe, which, you know, when you have a duty of care like Signal, again, this is not a game. This is not sort of a casual, like, oopsie, learning as we go, a bunch of data source spilled, there's a number of human rights workers now not able to get out of a country. Right? [00:15:14] S1: Right.
00:15:14 · Neil Chilson, Meredith Whittaker, Jeff Berman
[00:15:14] S1: Right. [00:15:14] S0: And I think this is this is not something where we can sort of hand wave around this. There are paradigmatic issues with the way agents are being built. Again, they are breaking down the walls we have put up to protect cybersecurity. The architecture of a lot of these agents, if you compare it to targeted malware, looks very, very similar. Get pervasive permissions on your device, root permissions, access to all of the data at the application level that comes through the device, hooking into screen buffers, using your accessibility tool kit, doing screenshots, running that through an optical character recognition system to create a database of everything that goes through your screen, then processing that, sending it off-site to process on third party serve... Off -site servers to, you know, get your inference, to send it back, and then driving your browser or any app as if it were you. Chase Bank just blocked Muse, because they cannot authenticate that it's an actual user using the bank with that extremely personal information, or whether it's an agent that's been prop injected to, like, take that information, and they see the liability concerns as too high. [00:16:26] S3: But they've blocked all agents. Right? They've not blocked all agents. [00:16:29] S0: Well, they've blocked the agent from the company that has a history of doing things at the, you know, the the most excessive level.
00:16:29 · Meredith Whittaker, Jeff Berman, Neil Chilson
[00:16:29] S0: Well, they've blocked the agent from the company that has a history of doing things at the, you know, the the most excessive level. [00:16:36] S3: Neil, this this sounds like then a market opportunity. This is a classic market opportunity. Right? There's an opportunity for a company to come in and say, we're we're the trusted agent. Maybe this is what Inflection will launch after today. Right? So, is it is it really just a question of one bad company or a question of not taking advantage of market opportunity and providing people with the safety and security that they want? [00:16:57] S1: Yeah, the incentives here are very different from the classic internet model that I was talking about. The incentives there were you were the product, and data was gathered not for your use, but it was gathered to sell ads, and that had a ton of benefits, but the the agentic model is using your data for you on your behalf and you're right, people will make poor decisions about how to do that, they will make poor decisions about how to deploy it, but companies will have a strong incentive to make sure that the types of bad things that you're talking about, companies are like, not all companies are quite as, adamant about such things as Signal, but companies have a profit incentive and they have an incentive deliver a product to their customers in this space that's quite different than the traditional internet. I think it's not our agents perfect, it's our agents and our internet with agents better than the previous internet and I think that's true.
00:16:57 · Neil Chilson, Jeff Berman
[00:16:57] S1: Yeah, the incentives here are very different from the classic internet model that I was talking about. The incentives there were you were the product, and data was gathered not for your use, but it was gathered to sell ads, and that had a ton of benefits, but the the agentic model is using your data for you on your behalf and you're right, people will make poor decisions about how to do that, they will make poor decisions about how to deploy it, but companies will have a strong incentive to make sure that the types of bad things that you're talking about, companies are like, not all companies are quite as, adamant about such things as Signal, but companies have a profit incentive and they have an incentive deliver a product to their customers in this space that's quite different than the traditional internet. I think it's not our agents perfect, it's our agents and our internet with agents better than the previous internet and I think that's true. [00:17:52] S3: So, why does Signal launch an agent? Give us an agent is the secure agent.
00:17:52 · Jeff Berman, Meredith Whittaker
[00:17:52] S3: So, why does Signal launch an agent? Give us an agent is the secure agent. [00:17:58] S0: I want to answer a different question, because we don't collect data, so we don't have data to launch an agent with, and we do one thing, and we do it really, really, really well, which is we provide verifiably secure and private messaging, so your communications are not in anyone else's hands. And we know that is, again, a matter of life and death for many, many people on... In a world where almost every other action communication that we have is gathered by a company, is used for marketing, is turned over to a government, is breached because some agent breached it. And, you know, privacy is not a nice to have for many people. [00:18:38] S3: I hear you that's not Signal's business. But but why not another company then? Why not a company where where safety and security, data privacy is at the core of what they're doing, and they're offering us an agent to make our lives easier where we can trust it with their information?
00:18:50 · Meredith Whittaker
[00:18:50] S0: Well, I think there are people trying to do that. But, again, we need to look at what is the promise, the marketing promise, that is being sold to people around these consumer agents. And it is something similar to having a magical genie servant that can do everything for you. Now, in order to do everything for you, it needs to know everything about you. And, yes, you can put harnesses on, you can add more sandboxes, you can have a sort of threshold that if it accesses this or that database, or is about to take an action, it has to check-in with me. But an agent that checks in at every step is kind of not an agent. An agent with no access to data doesn't quite work to fulfill that promise, and an agent that doesn't have access to all of your applications isn't really able to do that. So there is an issue here, paradigmatically, with what an agent is and what it does that I think gets in the way of some of that. And we... You know, again, cybersecurity could not be more prominent right now. I've been talking about this for about two years. We now have sort of, you know, the proof points are blowing up everywhere. And I do think we need to walk back some of this hype, some of this this marketing, and just get back down to technical fundamentals. Right? If an agent is screenshotting everything on your device, keeping that in a database, processing that off-site, you have just broken Signal's security model. You are eliminating the ability for privacy at the application layer. And, you know, I think in in contradiction to your statement agents give us agency, I see this move as pulling agency back to the AI companies, to the big OS vendors, as they grab data from independent developers, from websites, as they're shutting down the APIs at the back end, this is a data grab. And Amazon's lawsuit against Perplexity's agent, because Perplexity is in their marketplace, gathering a huge amount of data that Amazon does not want to share with a third party shows that this is actually a struggle between companies for data very much in line with the traditional Internet business model.