Masters of Scale
00:56:51 · Neil Chilson, John Donvan, Meredith Whittaker
[00:56:51] S1: I I I... Like I said, things will go wrong and people will learn. I think... [00:56:55] S2: Meredith, let let let's just look a little bit towards potential solutions. What is your... How would you want to address the issue? I mean Neil is saying that his great concern is that a move against agents would shut down a development that could help a lot of people. That's a strong part of his argument. I'm not sure if that's what you wanna do. Do you wanna stop the development of agents or do you wanna improve their use, which is what his argument is, that they can be improved. [00:57:20] S0: Well, the proposition that we are debating is should we trust them as they stand [00:57:25] S2: or I agree, but I'm moving
00:57:26 · Meredith Whittaker
[00:57:26] S0: beyond it. I get it. Get we're sort of water skiing through many many oceans right now. No, absolutely not. I think, you know, technologies can have, you know, cases that are you know, useful to be tautological about it. But I think it's, you know, we need to recognize that right now we have a lot of companies that are... Have spent billions and billions of dollars on developing AI. The CapEx versus revenue for the AI industry is a kind of like open closed secret. It's a little nerve racking to look at the market and the debt structuring of, you know, large scale AI right now. And there is a deep desire to be an AI leader and to find a consumer market fit for technologies that have, you know, cost a lot and on which so many lofty promises are riding. And so, you are seeing a rush to reckless deployments, to integrating agents into operating systems, to Meta Muse which, you know, a lot is banking on that. Their compute spend is high watering right now. And we are seeing the fundamentals of cyber security being cast aside as if we, you know, all just sort of, you know, got struck by lightning and forgot that these run on software and hardware, that our supply chains are buggy, that they require sort of, you know, infrastructures that at the network layer, at the software layer, at other layers of the stack can be hacked and perturbed. And that to actually use these systems, we're not talking about a new godhead strike, you know, that has come down from the clouds as a sentient machine learning system. We're talking about software and hardware infrastructures that are being integrated in ways that are deeply unsafe. And now if I'd said that three months ago, I would have to do some explaining. But now we have the hugging face incident. We have Microsoft's Copilot exfiltrating data. We have prompt injection attacks all over the place and we have a week where the PHP, you know, supply chain issues, the the bug in PHP means that 75% of websites are vulnerable in ways
00:59:41 · Neil Chilson, John Donvan, Meredith Whittaker
[00:59:41] S1: that Bug identified by them. [00:59:42] S2: So what I what I hear, Neil, from Which [00:59:44] S1: is not... [00:59:44] S0: You know, like, I'm not arguing that AI is bad. [00:59:46] S2: Right. I I I [00:59:47] S0: am making a high stakes argument that the way this is being rolled out is extremely perilous to the infrastructure, the, you know, the computational infrastructure, the network infrastructure, the Internet, that is at this point the nervous system of core institutions, our governments, our, you know, critical infrastructure Right energy [01:00:05] S2: now, you're saying don't trust because of what you called a reckless rush. So Neil... [01:00:09] S0: I'm saying it's not trustworthy. [01:00:11] S2: Not trustworthy now. You're not saying it can't ever be, but you're saying right now, if you look at the language of the proposition, it's not trustworthy because of what Meredith Neil is describing as a reckless rush. That this this is a bad moment in terms of what's going on. What what's your response to that? If you're not arguing for the future, but you're arguing for this moment. [01:00:29] S1: Well, I think the the interesting examples, you know, Meredith said earlier that coding was one of the areas where these are extremely useful. And all the examples that she gave of the the biggest threats are also in that space. And so this is the first frontier, this is the space where we are trying to figure out how do we balance this properly. How do we deploy this technology so that defenders have the advantage over the offense? And...
01:00:29 · Neil Chilson, John Donvan
[01:00:29] S1: Well, I think the the interesting examples, you know, Meredith said earlier that coding was one of the areas where these are extremely useful. And all the examples that she gave of the the biggest threats are also in that space. And so this is the first frontier, this is the space where we are trying to figure out how do we balance this properly. How do we deploy this technology so that defenders have the advantage over the offense? And... [01:00:57] S2: But are are you not con... Do you not share some of the deep concerns he's talking about this moment that we're in? [01:01:01] S1: Of course. Of course. [01:01:02] S2: You do. [01:01:03] S1: Of course. We need to do this right. And so the question is, do we have the right structures and incentives in place to make that happen? [01:01:11] S2: Do you think so? [01:01:11] S1: And I think there are many things that we can improve on. I think there should be a lot more coordination between the companies. I think we need a lot more research into this space. I think we also need to make sure that companies aren't, you know, using this language of agents to to get off the hook. That, oh, it was our agent who did this and therefore, it's not... It's kind of out of our control. That's that's not fair, and I don't think under the law that that stands up. So we we do need responsibility for these companies, for the damage that their agents cause, and I I think I think the current law is there, but there are some gaps to fill for sure.
01:01:11 · Neil Chilson, John Donvan
[01:01:11] S1: And I think there are many things that we can improve on. I think there should be a lot more coordination between the companies. I think we need a lot more research into this space. I think we also need to make sure that companies aren't, you know, using this language of agents to to get off the hook. That, oh, it was our agent who did this and therefore, it's not... It's kind of out of our control. That's that's not fair, and I don't think under the law that that stands up. So we we do need responsibility for these companies, for the damage that their agents cause, and I I think I think the current law is there, but there are some gaps to fill for sure. [01:01:48] S2: Before we wrap, if you'll indulge me, this is the first time we've ever done a post game show on a I'm just curious for each of you what the experience of being in a debate was like. I don't know if either of you have been doing a lot of debating or not. Both of you shaking your heads no. Except on Twitter, yeah. Meredith, did anything about the experience when you were up there surprise you?
01:01:48 · John Donvan, Meredith Whittaker, Neil Chilson
[01:01:48] S2: Before we wrap, if you'll indulge me, this is the first time we've ever done a post game show on a I'm just curious for each of you what the experience of being in a debate was like. I don't know if either of you have been doing a lot of debating or not. Both of you shaking your heads no. Except on Twitter, yeah. Meredith, did anything about the experience when you were up there surprise you? [01:02:12] S0: No. I do a lot of public speaking, so, you know, but the thing... I don't really care about the debate. I'm not trying to win points. I am really concerned and alarmed by the technical landscape I see right now. I am very concerned that a lot people, you know, maybe in search of money, maybe because it's just a... There's a intoxicating AI fog that we're all living in, have... You know, people who taught me technical fundamentals seem to have forgotten them in service of a a story about, you know, superhuman machines that can do everything for us. [01:02:50] S2: But to my point, was just more public speaking for you. Not not the sense What of being in a [01:02:56] S0: I wanted was to ensure that I was clear, that I acquitted myself honorably, and that the stakes of something that can seem arcane and technical and beside the point to normal people who have laundry to do were clear. [01:03:09] S2: Neil, what about you? Anything surprise you about the experience? And I I I note and we chatted earlier that while there were applause lines throughout the debate, most of them seem to be going to your opponent. [01:03:18] S1: That's true. That's true.