00:07:18 · Thai Duong
[00:07:18] S0: Oh, that's a good question because I think in the past, just the recent past, like such an exploit was very difficult to develop. You need the vulnerabilities. You may need like multiple vulnerabilities, not just a single one. You need a way to get into the phone and then you need to get to root, like it... Like, basically to compromise the entire phone. And usually, it start with, like, you know, you visit a website or you receive a message on, you know, WhatsApp, on Signal, or on iMessage. And then the message or the website would exploit vulnerability in these apps. And then it would try to go from that vulnerability to the kernel of the phone, and that's how it gonna be able to control the entire phone. And you can see that's very difficult to do. Right? It it requires multiple vulnerabilities. It usually require a very well funded team of like several people to work very hard for a few months to be able to come up with such a vulnerability and exploit. So it's very expensive. And only a few groups of people all over the world can actually do it. And that's why this category, somebody have it, they don't want just to use it randomly because it's so expensive to develop. So AI changed that equation a little bit. Because now with AI, things become easier. Of course, you still need the human experts. You still need people who understand the iPhone architecture and everything. And then with the right AI model, frontier models, they would be able to speed up the vulnerability discovery and exploitation. In our team, for example, it took us like two or three weeks to develop these chains, and we have multiple chains. And so we believe moving forward, there will be more people having this capability. And it raised a lot of question about the privacy and everything for everybody, right? Because imagine if your neighbor can just hack you by sending you a message, what does it mean for your privacy? And I think hopefully, because we find these bugs and exploits to fix them. And we are working with, like, the vendors, in this in this case, Apple, to get them fixed. So hopefully, like, a lot of these bugs would be killed before these capabilities get into the hands of, like, everyday... Everyone. So, yeah, that's basically the situation that we are in right now.
00:10:27 · Ruby Justice Thelot
[00:10:27] S1: Right. What do you think about the potential for self replicating agent swarms or botnets that are able to take over much of the Internet? Dario Amodei warned about this. He said, I believe it was about two months ago at this point, he said within six to twelve months, we might have agent swarms that are capable of doing this. So what do you think?
00:10:51 · Thai Duong
[00:10:51] S0: So we have seen worms before, right? We've seen WannaCry, for example. And in the summer, we also published something we call WeWorm. It's basically a vulnerability data that we file in WeChat that we can just call somebody. The victim doesn't need to pick up the phone and after a few seconds, we can hack their phone. And then we use their phone to call their friends and that's how we worm can spread over the WeChat network. And so it's a possibility. In the past, I think the people who have the capabilities to develop these kind of worms, they tend to be very discreet, right, because they want to use the capabilities for targeted attacks. They don't just want to make noises. Now because of the capabilities could be in the hands of many more people, there may be accidental leak. People may try to develop a worm in a lab and there's a lab leak and the worm leaks and spread to everybody else. And I'm talking about like normal computer worm and I believe you asked about AI worm, like the agents, they want in an effort to maybe stay alive, just try to replicate itself. And I think this is a possibility, right, because the models, if you think about it, it's just a bunch of files, just numbers, right? And right now, the models could be maybe a few 100 gigabytes. But we also know that they are smaller and smaller models. And from the AI point of view, like it knows that when the humans are upset with it, they can try to like to shut it down. So it's very natural AI to think about, okay, the first step is to stay alive and to replicate ourselves into multiple machines. And you can see the neo cloud providers out there, they don't have very security. If you saw recent news, there's a lot of like break ins or incidents at these neo clouds. And the neo clouds are perfect location for a model to hide itself, right, because the cloud has GPUs, has a lot of compute. And the cloud by definition holds models. So if I... It's so weird, but if I'm were a model, I would try to target these neo cloud. And we can see this, they not very difficult to hack. So I wouldn't be very surprised if one day we'll find maybe the models don't intentionally do that. It doesn't intentionally try to replicate itself. It just tries to accomplish some goal that some human asks us to do.
00:14:11 · Thai Duong, Ruby Justice Thelot
But during that process, it realized that the first step is to have more agents helping it and then it needs to replicate itself. And then it try to hack some neo cloud providers to replicate itself, like not because it's want to stay alive but because it just want to complete... Accomplish or finish some task. I think nothing may happen before like the sci fi scenario where AI become, like, fully sentimental and try to hide from humans. [00:14:50] S1: Right. Why are NeoClouds easy to hack?
00:14:54 · Thai Duong
[00:14:54] S0: Oh, wow. Because their business model is like this. They give you... They give their customer machines. Right? Raw bare metal machines. And these machine plug into their networks, and they run, you know, a lot of software that usually people like us don't have access to. So the vulnerabilities in the software are there, but nobody, you know, have a chance to look at them before. I can give you an example. For Neo Cloud, they're going to give you, for example, you sign up for Neo Cloud. They would give you a physical machine. And then this physical machine, it has like three or four networks. The first network is the network, the main network for the physical machine. And then there's another computer on a physical machine that's called VMC, which is like how you can manage the machine remotely. And this is another network, another interface that you can actually compromise the machine. And then the machine also connects to the AI GPUs. And usually it connects to these GPUs via something called like InfiniBand network. Also manufacture... Met by... And zoned by NVIDIA. And then InfiniBand basically is another network, another attack by surface that the cloud... Neo cloud providers has to worry about. And then at the end of the line would be the GPU themselves. So you have like so much attack surface that the attack surface... Like there's one of the most famous researcher in this space is Mark Dowd. And he said this, like the attack surface is the vulnerability. If your attack surface is very large, then being compromised, attacking you is just a detail. So the Neo Cloud basically just exposed so much stuff, and none of this stuff got enough attention from security professionals because... It's just simply because these stuff are so expensive to acquire. If you really want to get into like data center security, you need to acquire GPUs. And these days, even if you have money, you cannot really just go buy GPUs. So that's the problem.:
00:17:23 · Ruby Justice Thelot
[00:17:23] S1: Right. So do you think that in the long run cyber will be offense or defense dominant? Like seems like... Actually, it seems like few people think that cyber will be offense dominant literally forever. Well, do you think that it will be? And if not, how long do you think it will take to get to a state of stable defense dominance?