"The Cognitive Revolution"
01:08:38 · Nathan Labenz
[01:08:38] S0: Earlier that Friday morning, before the guests, a new benchmark had been making the rounds. AIs playing blood on the clock tower, a social deduction game of bluffing and collusion. And the timeline was cheering how good they have gotten at it. Here is what I made of the game. If you kind of take it in isolation, it's like, you know, all else equal, it is interesting to know how good are the AIs at playing this game. Are they getting good at deception? Are they getting good at collusion? Are they getting good at sniffing out when other people are trying to deceive them or collude against them? These are all, like, interesting questions. What kind of bothered me about it, though, was the sense that, like... And then Noam Brown comes in over the top and says, wow. This is great work. You know, you wanna come here and make evals? And that kind of embodied to me, like, what I thought was problematic about this whole thing, which is just like, we're just gonna hill climb on this now. Right? Like, it it seems like the the mentality of the field as a whole is like, oh, isn't this exciting? You know? And let's have a horse race as to who can be the best blood on the clock tower AI player. And I think what we're all kind of forgetting as we get all wrapped up and excited about this and, you know, wanna go play the AIs ourselves and everything is like, this is not good behavior from the AIs. Like, we do not want AIs deceiving and colluding. I'm sure all these, you know, people have sort of a thoughtful response to this, but it's just like zoom out and the situation is bad. You know, we we have AIs that are untrustworthy in all these kinds of ways, and we're not even, like, taking that beat to say, hey. Is this what we want? Do we want AIs that are gonna outconnive and outcollude us? I don't think we want that. That's, like, exactly... You know, this is very similar to my idea that we should not train AI generals. Like, let's not create... No. Let's not train AI master manipulators. It's so simple, but here we are. And, like, I didn't hear... I didn't see... I saw a lot of cheerleading and a lot of enthusiasm, and I did not see any corrective commentary on the timeline saying, hey. Is this really what we want? Back to the end of that same Friday show after Bryan had signed off, Prakash with a picture of where the economy goes from here.
01:11:02 · Prakash Narayanan
[01:11:02] S1: So I feel like the going forward economy is an agentic economy, and that means agents building businesses and software for other agents, which are suitable for their affordances, which is going to be massively parallel, high speed, able to absorb information and distribute information at very low latencies. We are going to end up with a Salesforce primarily used by agents, a ramp primarily used by agents. All of these softwares are Stripe primarily again used by agents. Maybe Stripe or other companies will be the ones that create the software. I am very doubtful. I'm very doubtful because I don't think they will move that fast. I don't think they are capable of moving that fast. I don't think they understand the affordances. And I know Stripe is basically people a billion times smarter than I am. I'm not trying to downplay how effective and how intelligent and how competent Stripe is. You're going to get 50,000,000 Patrick Collison. You're going to get 50,000,000 Elon Musk's. You're going to get humans who give them capital to deploy. You're going to get agents coordinating investments into businesses built by agents for agents. And so I largely see the future of the economy as primarily agent driven with us just receiving free benefits. Because hey, CRM call from a human is so costless in comparison to the massively parallel CRM calls that an agent would make. The abundance basically comes from us building all this stuff for this agentic economy. As a result, we just have all of this excess because we just don't consume that much. We're just not that consumptive in comparison to what the agents are gonna consume. And and that's and that's where this abundance is gonna come from.
01:13:02 · Nathan Labenz
[01:13:02] S0: My reply leans on a side project I'd launched the week before, Fruitopia, which finds the best fruit at your local grocery and around the world. Here's where I came down on the economy Prakash had just described. And on the one hand, it's like cool if, you know, there's abundance in, like, API calls because the agents are using so many and they, like, build up, you know, great systems that they can use, and then we get to kind of piggyback on them. But, like, there's a lot more to life than API calls, and I do feel like we're sort of... Especially... And especially those of us that are in the AI space are, like, really quick to forget that, or we sort of, you know, are... We're kind of contorting ourselves in a way to, like, like this future? This is why I've been, like, holding myself to this idea of, like, I wanna be getting outside more and getting more exercise. And if I'm not doing those things, am I really winning, or am I just kind of plugging myself into some, you know, AI powered machine as, like, some sort of, you know, human integration layer type, you know, component that's, like, in a way, like, too subservient to the machine. But this is why I decided to do this Fruitopia project recently. And, you know, it's again just one of these things where it's like, okay. If we are in a bright future, then I think one, you know, very kind of deeply rooted thing that we all like, we're evolved to like, is eating delicious, fresh, in season fruit. Right? Like, everybody loves that. It's it's it's... Goes back to when we were monkeys that, you know, we loved that. When I tweeted that, I got, like, basically no play on it versus the blood on the watchtower thing. Everybody's like, oh my god. Like, look how great AIs are getting at, like, deception and conniving and, you know, winning these, like, games of social manipulation. And I'm just like, I don't really want that from my AIs. What I want is a future where I have not abundance of API calls, but abundance of things that are, like, more biological. Give me the fruit. You know? That's what I really want. The the fruit of the AI revolution should indeed be, like, at least in part, actual literal fruit. And if we're losing that, we're, like, losing so much of what has mattered to us.
01:15:21 · Nathan Labenz, Prakash Narayanan
And if we can't even, like, notice that we're losing it, I think in a way, you know, that that sort of worries me and makes me think, like, maybe we've kind of already lost the plot. [01:15:32] S1: I mean, it's pretty obvious to me that we're gonna get that grown, like, food, which is far more nutritious and far more tasty. It's gonna be, in the twentieth century, we discovered chemicals. And then when you look at these food products right now from Doritos or whatever, those are so scientific. There are like thousands of PhDs who have worked on Doritos. Thousands. Literally thousands. The old... [01:15:59] S0: Remember the old onion article? Doritos celebrates its one millionth ingredient? [01:16:03] S1: Like, they have figured out your flavor palette so well that they can pinpoint exactly what flavor that they want to produce. And all of that technology is going to get deployed into bio. We're going to produce fruits and vegetables, etcetera. Hybrid fruit is already basically GMO ed, but we just, you know, took a very long time to GMO it. Part of abundance is gonna be, you know, your 99 percentile wine available for $2 chuck. Right?
01:16:32 · Nathan Labenz
[01:16:32] S0: My favorite apple is the Honeycrisp, and, you know, it's a relatively recent creation. Not GMO as far as I understand, but definitely the product of, like, large scale systematic crossbreeding, which is sort of, you know, GMO before there was GMO. So, you know, I... I'm not afraid of GMO enhanced fruit by any means. I'm all for it. But but it is still something that is, like, a biological construct that feeds us as biological creatures and, like, you know, delights us in terms of our biological senses and the, like, you know, ephemeral qualia that we get from them. And, like, stuff is just all too often lost because it's not very measurable. It's not so hill climbable, and it's not what the agents need. Right? And I think we we are gonna need to be very careful not to create an economy that really serves agents and creates abundance for us in things that are core to what agents need. We're gonna... If if we that's really what... I kind of agree with you. That might be what happens by default, but I would challenge everybody to think, like, wait. Like, what if the stuff that agents need in abundance isn't the same stuff that makes our lives good?
01:17:46 · Prakash Narayanan, Nathan Labenz, Mike San Miguel
[01:17:46] S1: One of the annoying parts is that, you know, the the Frontier lab guys are very serious. Right? And so when you're very serious, you only talk about cancer, curing cancer. No one's talked about a better Doritos. No one's come out and say, like, we're gonna make a better football game. Where are these? We're going to make a better burger. We're going to make a better football game. Where are these things that people enjoy? I think that's because they want it to be serious. They want to show that earnestness. They don't wanna come off as foolish, etcetera. Maybe you should focus on, we're gonna make a better Doritos chip. We're gonna make one that's healthier for you. We're gonna make a Doritos that is naturally grown from the ground, and you just need to cut open the potato and it's Doritos. And and this is like... This sounds like so frivolous and idiotic, but this would catch the imagination of, like, people on what is actually gonna be possible. Right? [01:18:37] S0: I mean, I think it's also because they do have this kind of World War two mindset about their work. You know, that they they genuinely do believe time is short, stakes are high. The... You know, their their own, like, allocation of time is, like, very skewed toward work. You know, there's just not a lot of space for leisure in the lives of the people at the frontier companies. [01:19:03] S2: So Yeah.