Masters of Scale
00:11:43 · Meredith Whittaker, Jeff Berman, Neil Chilson
And so this, again, I don't care about winning this debate, I care about looking this in the face and recognizing we are eating the seed corn of our cyber security when we install these things in the name of convenience without thinking through the perilous architectures that we have used to create them. [00:12:03] S3: So Neil, given the check... Given the given the challenge of segmenting and locking down data, why is the convenience value worth more than the risk we're creating here? [00:12:17] S1: Well, I I think it's easy for us to dismiss these types of challenges, but again, the the goal here is not that agents are perfect or that they're perfect protectors of security. The question is, how do they compare to the status quo? And the status quo is not great. Most people don't use Signal, unfortunately. And the status quo is full of people who are gathering information about us in part because we've made that convenience trade off over and over and over. So Meredith is totally right that we need secure agents. We need better agents, but the challenge is not to not use. Remember what the proposition is. The proposition is, should we trust agents with our information And the... And... Or should we not trust agents? And it's a conditional. It depends. It really depends on the agent. It really depends on the use and it really depends on the access that it's asking for. Most of the agents right now, I don't let them write online, I let them read online for the most part.
00:12:17 · Neil Chilson, Jeff Berman
[00:12:17] S1: Well, I I think it's easy for us to dismiss these types of challenges, but again, the the goal here is not that agents are perfect or that they're perfect protectors of security. The question is, how do they compare to the status quo? And the status quo is not great. Most people don't use Signal, unfortunately. And the status quo is full of people who are gathering information about us in part because we've made that convenience trade off over and over and over. So Meredith is totally right that we need secure agents. We need better agents, but the challenge is not to not use. Remember what the proposition is. The proposition is, should we trust agents with our information And the... And... Or should we not trust agents? And it's a conditional. It depends. It really depends on the agent. It really depends on the use and it really depends on the access that it's asking for. Most of the agents right now, I don't let them write online, I let them read online for the most part. [00:13:14] S3: And And I so think Meredith's it's all about those saying, trade then you've got risk of prompt injection. Right? You've you've got concerns on top of concerns. So if we can't segment and lock down data and we have concerns about prompt prompt injection, why does the convenience outweigh the risk?
00:13:14 · Jeff Berman, Neil Chilson, John Donvan
[00:13:14] S3: And And I so think Meredith's it's all about those saying, trade then you've got risk of prompt injection. Right? You've you've got concerns on top of concerns. So if we can't segment and lock down data and we have concerns about prompt prompt injection, why does the convenience outweigh the risk? [00:13:27] S1: So, I think the only way we get to solve these problems is the same way that we solve problems in messaging, right? We didn't give up on messaging. We didn't stop messaging because there were risks and there were there were platforms that were built poorly. We built better platforms. Meredith was a key part of that, and so I think that that's what we need to do here. We're going to learn trust and we're going to learn how to improve these products by using them, not by standing back and letting people who don't care about privacy build them. [00:13:57] S3: Meredith, one of the points I heard Neil make in his opening is, [00:14:01] S2: to your [00:14:01] S3: point about us being privileged people, I'm guessing you have an executive administrative assistant, many of them in the room do. They probably have access to an enormous amount of your life, your contact information, etcetera. Most people can't afford to have someone do that. And this is a massive time saver, a massive efficiency creator for people who don't have the means otherwise. What's really the difference between you're having a human admin and you're having a digital admin?
00:14:01 · Jeff Berman, Meredith Whittaker, Neil Chilson
[00:14:01] S3: point about us being privileged people, I'm guessing you have an executive administrative assistant, many of them in the room do. They probably have access to an enormous amount of your life, your contact information, etcetera. Most people can't afford to have someone do that. And this is a massive time saver, a massive efficiency creator for people who don't have the means otherwise. What's really the difference between you're having a human admin and you're having a digital admin? [00:14:27] S0: Well, Emma, my assistant, is incredible, but if Emma messed up like this, I would fire her. There is labor contract that protects my relationship with Emma. She understands, as a human being, how sacrosanct the information that she has access to is, and she's accountable for it. She's not a bot that is installed at a pervasive level, who's sort of fumbling through text, you know, that that could lead to a serious catastrophe, which, you know, when you have a duty of care like Signal, again, this is not a game. This is not sort of a casual, like, oopsie, learning as we go, a bunch of data source spilled, there's a number of human rights workers now not able to get out of a country. Right? [00:15:14] S1: Right.
00:15:14 · Neil Chilson, Meredith Whittaker, Jeff Berman
[00:15:14] S1: Right. [00:15:14] S0: And I think this is this is not something where we can sort of hand wave around this. There are paradigmatic issues with the way agents are being built. Again, they are breaking down the walls we have put up to protect cybersecurity. The architecture of a lot of these agents, if you compare it to targeted malware, looks very, very similar. Get pervasive permissions on your device, root permissions, access to all of the data at the application level that comes through the device, hooking into screen buffers, using your accessibility tool kit, doing screenshots, running that through an optical character recognition system to create a database of everything that goes through your screen, then processing that, sending it off-site to process on third party serve... Off -site servers to, you know, get your inference, to send it back, and then driving your browser or any app as if it were you. Chase Bank just blocked Muse, because they cannot authenticate that it's an actual user using the bank with that extremely personal information, or whether it's an agent that's been prop injected to, like, take that information, and they see the liability concerns as too high. [00:16:26] S3: But they've blocked all agents. Right? They've not blocked all agents. [00:16:29] S0: Well, they've blocked the agent from the company that has a history of doing things at the, you know, the the most excessive level.
00:16:29 · Meredith Whittaker, Jeff Berman, Neil Chilson
[00:16:29] S0: Well, they've blocked the agent from the company that has a history of doing things at the, you know, the the most excessive level. [00:16:36] S3: Neil, this this sounds like then a market opportunity. This is a classic market opportunity. Right? There's an opportunity for a company to come in and say, we're we're the trusted agent. Maybe this is what Inflection will launch after today. Right? So, is it is it really just a question of one bad company or a question of not taking advantage of market opportunity and providing people with the safety and security that they want? [00:16:57] S1: Yeah, the incentives here are very different from the classic internet model that I was talking about. The incentives there were you were the product, and data was gathered not for your use, but it was gathered to sell ads, and that had a ton of benefits, but the the agentic model is using your data for you on your behalf and you're right, people will make poor decisions about how to do that, they will make poor decisions about how to deploy it, but companies will have a strong incentive to make sure that the types of bad things that you're talking about, companies are like, not all companies are quite as, adamant about such things as Signal, but companies have a profit incentive and they have an incentive deliver a product to their customers in this space that's quite different than the traditional internet. I think it's not our agents perfect, it's our agents and our internet with agents better than the previous internet and I think that's true.