Masters of Scale
00:04:38 · Demis Hassabis, Reid Hoffman
[00:04:38] S2: Yeah. For sure. I mean, we take a step back for a moment, it's like big picture, and you know this knowing us from quite early in our DeepMind journey, that the whole point of me working on AI for my whole career was to use it as a kind of like the ultimate tool to advance science and medicine and our understanding of the world around us, but also improving human health. I've always said that the number one thing we should use these amazing tools that we've now built, and we were, you know, dreaming about twenty years ago, is to improve the human condition and improve human health. And I've tried to do that certainly with my own more personal projects, obviously things like AlphaFold, but also the spin out drug discovery company, Isomorphic. I've tried to push it really hard in that direction, as I know you have too more recently. And I think it's exactly the sorts of things we need to be showing the world that AI can do. I think what better thing than to use AI to help us cure terrible diseases. [00:05:34] S3: So some of the protein folding was this kind of intersection of the idea of, like, we have really good data. We could use scale learning because the whole learning... The cognitive revolution from learning starting with even Pong, but the... You know, and self play and all [00:05:50] S2: the rest
00:05:50 · Demis Hassabis, Reid Hoffman
[00:05:50] S2: the rest [00:05:50] S3: is like, okay, that's... This would be a really good way to do it. How does AlphaFold lead to medicines? Because one of the things that is a question I get frequently is like, yeah, you're talking about that, but is it true? Is it going to happen anytime soon? So I'm passing along the question I get to you.
00:06:05 · Demis Hassabis
[00:06:05] S2: Yes, sure. Well, biology is the most complex systems in the universe. That's why it's so hard because of the emergent behaviour, the interactions, and proteins are the building blocks of life. That's how I've always seen them. The protein folding problem, which is what we tried to solve with AlphaFold, was considered to be like a fifty year grand challenge in biology. It's sometimes described to me as like Fermat's Last Theorem, but for biology. It's critical to understand the structure of a protein from its genetic sequence, its amino acid sequence, to understand its function. And that's really important to understand basic biological mechanisms, but also to understand what goes wrong in disease and what things to target. You know, if you imagine you're designing a drug, then you need to know what the three d surface of the protein looks like so you know which part of that protein to bind to. And so it was always a critical, but not obviously only still a small part of the entire drug discovery process. But it's an absolutely critical part of that. And we were able to, you know, with AlphaFold, accelerate it by orders of magnitude by building AlphaFold. And interesting is that at the time... And if you think about it, we finished AlphaFold two back in 2020, which is, six plus years ago now, and it's still probably the best example of... Concrete example of AI applied to a really hard scientific problem that was thought to be sort of uncrackable. That just shows you how far ahead of its time it was, And now, you know, we're really proud of of the impact it's made. Like, 4,000,000 researchers around the world use it. Pretty much every biology and medical researcher in the world, almost every drug in future will have made use of AlphaFold, all the structures. But it's not enough. We want to expand it to biochemistry and also predicting things like toxicity or acne properties, all of these other things that you need to know about before you can have a drug that's safe to use and that's efficacious you know, efficient in whatever it is you're trying to tackle. Yep.
00:08:02 · Reid Hoffman
[00:08:02] S3: So speaking of accelerations, let's go to one of the hot topics of the day, recursive self improvement. Mhmm. What is your kind of theory of the game? And I say this because you, you know, are a master games person. So I use that as kind of deliberate. And if you don't know what I'm referring to, either look at the documentary or look it up on Wikipedia. So like RSI, opportunities, concerns? Yeah.
00:08:28 · Demis Hassabis
[00:08:28] S2: Well, look, RSI, it's obviously an extremely useful technique to make progress. It mostly works best in what I call verifiable domains, what's called verifiable domains. So math is one, which is why we're seeing so much advance there, coding's another, but originally it was games. So actually, one of the first sort of famous recursive self improving systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which is the kind of, you know, famous program that beat the world champion of Go back in '20... You know, ten years ago now, 2016, and kind of was the beginning, I think, of the modern AI era. And then we built a second version that was in some ways more consequential, AlphaZero, that didn't need to use any human data anymore, and, it just basically played against itself. Started at random, and then after, you know, around a dozen to two dozen different epochs, you know, it played like a million games against itself. It would then train a new version of itself that was slightly better than the old version, and then that would play a million games against itself and so on. And for example... And then it could play any game, not just Go. It could play any two player per information game. And for me, it was especially visceral as a chess player because I remember watching it play chess. And within, you know, twelve hours, basically, or less than twelve hours, it would start from random play in the morning to, you know, you'd have your coffee break at eleven a. M, and I could still just about beat it. Then by lunchtime, it was Grand Master level, and then by tea time, it was better than any chess playing entity.
00:10:06 · Reid Hoffman
[00:10:06] S3: Tea time is dinner time for The