Masters of Scale
00:08:28 · Demis Hassabis
[00:08:28] S2: Well, look, RSI, it's obviously an extremely useful technique to make progress. It mostly works best in what I call verifiable domains, what's called verifiable domains. So math is one, which is why we're seeing so much advance there, coding's another, but originally it was games. So actually, one of the first sort of famous recursive self improving systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which is the kind of, you know, famous program that beat the world champion of Go back in '20... You know, ten years ago now, 2016, and kind of was the beginning, I think, of the modern AI era. And then we built a second version that was in some ways more consequential, AlphaZero, that didn't need to use any human data anymore, and, it just basically played against itself. Started at random, and then after, you know, around a dozen to two dozen different epochs, you know, it played like a million games against itself. It would then train a new version of itself that was slightly better than the old version, and then that would play a million games against itself and so on. And for example... And then it could play any game, not just Go. It could play any two player per information game. And for me, it was especially visceral as a chess player because I remember watching it play chess. And within, you know, twelve hours, basically, or less than twelve hours, it would start from random play in the morning to, you know, you'd have your coffee break at eleven a. M, and I could still just about beat it. Then by lunchtime, it was Grand Master level, and then by tea time, it was better than any chess playing entity.
00:10:06 · Reid Hoffman
[00:10:06] S3: Tea time is dinner time for The
00:10:07 · Demis Hassabis
[00:10:07] S2: US folks. Yes, computer time. Or computer, know, kind of handcrafted chess computer is better than all of them. So we saw the power of that recursive self improvement and obviously the thing that all these domains have in common is you can verify the answer. Is it a correct answer? Is it an optimal answer? Is the equation correct? Does the code run? Did you win the game? So those domains are... And and and the important thing about a verifiable domain is you can verify the answer, so you know when you've you've you've you've the right point. But, also, that means you can create almost infinite synthetic data because you can just use the current best versions you have to set new problems or make new moves and then just sort of verify whether the synthetic data was correct. The problem with synthetic data a lot of the time is you can't verify that it's right from the right distribution, but in those cases you can. So we've seen the enormous progress with maths and with coding, and coding's obviously become enormously valuable, and everyone is having to do that now. But I do worry about it because in the limit... So there's different forms of RSI. Right now, we're in the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is, you know, it's making a human engineer maybe three, four, five times more productive. Okay? So... But the but the human is very much still in the loop and reviewing code and architecting things and and writing code still, but it's all just enhanced. The next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure. That will obviously be faster, but I think is also a very dangerous way to get to full AGI because we're likely for two reasons. It sometimes used to be referred to twenty years ago as like an intelligence explosion because you don't quite know how do you control, keep control of such a system, such a process. And then the end result will likely be quite alien. It's gonna be like an alien intelligence. There's no reason you... To think it would be sort of in the similar kind of mind space that humans are in. So that will make it very hard for it to understand... To...
00:12:23 · Demis Hassabis, Reid Hoffman
For us to understand it and maybe for it to understand us. So I think it's... You know, we should be very cautious as we get closer towards that closed loop form of RSI. And I think all the AI researchers sort of know that, but there is a sort of race to the there's this sort of intense, obviously, competitive race to improve your coding systems. [00:12:43] S3: So presume that the incentives to be in a race continue, and you and I, some people already know this, have tried to figure out if there's various ways you can get global collaboration to have less. You've dedicated a huge amount of energy to that, so thank you. It's a hard game Mhmm. As as you know. But presume that the competitive pressure for continuing to do RSI and coding continues. In that environment, what are the best ways that we can potentially navigate that? Because we have to see kind of where our SI is going, what it's pointed at. What are the kinds of things that the, you know, the leading labs such as your own Yeah. And the others, what should they be doing Yeah. To try to navigate this
00:13:27 · Demis Hassabis
[00:13:27] S2: Well, what we're doing, and I think other leading labs are trying to do, is you've got to... So there's more prosaic things like make sure cyber defense is accelerating faster than cyber attack and making sure you give access to trusted cyber defenders first on maybe slightly fine tuned versions of your general models that are fine tuned for cyber defence. And so you keep the defenders ahead of the attackers. It's a pretty tricky problem to do in general. Also, building much more hardened sandboxes. I think a lot of the cyber kind of issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. I think better, you need really amazing monitoring systems. You know, they shouldn't be possible for agents to be running for for weeks without anyone realizing. You know, these kinds of things that we've seen some of our some of our competitors and others, maybe, you know, it's just the beginning of this happening. So I think everyone will get rapidly better, but they have to as well because otherwise, something bad's gonna happen inadvertently, some kind of incident, you know, I sometimes call it like an oil spill, is going to happen on the current trajectory if we're not if we don't get collectively, if everyone doesn't pay a lot more attention and put a lot more resources on these types of systems. Eventually, you kind of want formal guarantees, maybe some bounds around what these systems can do and how robust test, how robust, like, the things like the guardrails are. This is actually something I encourage a lot of top mathematicians that I know well who are maybe, you know, pretty sad about the state of pure maths, but they... Their skills would be incredible if they pivoted to these types of domains, more, like complexity theory, game theory. How do you, and, of course, we're not gonna have to start dealing with swarms of these agents, perhaps potentially collaborating in some way. How does one keep track of all of that?
00:15:14 · Reid Hoffman, Demis Hassabis
[00:15:14] S3: Should there be ways that the labs are encouraged, required, permitted to collaborate on this? And if so, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology, mutual accountability. [00:15:31] S2: Yeah. Mean, been lots of proposals, I've been working on these things a lot. You saw my essay over the summer, maybe some of you read it, about potential for like an SRO, industry run standards body with some kind of government oversight over it. I think something like that is going to be needed, then ideally it becomes international, you know, it's US led, and then obviously the state of geopolitics today makes it, and and our international institutions are very, very weak as we all know, just at the time when we need them to be strongest in a way. So that's a bit unfortunate, but we've got to start somewhere. [00:16:04] S3: By the way, British, a bit unfortunate is holy just to translate...