"The Cognitive Revolution"
00:01:05 · Nathan Labenz, Mike San Miguel, Brian Zhou
[00:01:05] S0: If you have political willpower, all contracts are worth dust. Mike San Miguel of Luxor Technology. He sells GPU servers off its hardware desk, and he set up its KYC department. [00:01:16] S2: We've heard there's some compliances. These things are traded, more closely than than some nuclear secrets. So every OEM is a little bit different. NVIDIA works closely with all the OEMs around allocation and around green lighting orders. NVIDIA also typically is gonna check every six months and make sure those GPUs are where they're supposed to be. Right? They'll physically come on-site and actually make sure that they haven't left. [00:01:42] S0: Brian Zhou, a researcher at Thoughtful. He helped build Ballot Bench. Nine frontier agents, each given a week to forecast the midterms, and trading agents putting real money on their forecasts. [00:01:54] S3: It takes forecasting agents a bit of time to say, see an election results, communicate it to their trader, etcetera. And by that time, more programmatic or more deterministic algorithms and bots have already traded on this information before. So this is, like, pretty indicative of what we were seeing where the traders were doing quite well the week before election day, and then on election day, they simply didn't react fast enough and sometimes made up incorrect information, which resulted in the traders losing a lot of money.
00:01:54 · Brian Zhou, Nathan Labenz, Bryan Bischof
[00:01:54] S3: It takes forecasting agents a bit of time to say, see an election results, communicate it to their trader, etcetera. And by that time, more programmatic or more deterministic algorithms and bots have already traded on this information before. So this is, like, pretty indicative of what we were seeing where the traders were doing quite well the week before election day, and then on election day, they simply didn't react fast enough and sometimes made up incorrect information, which resulted in the traders losing a lot of money. [00:02:24] S0: Bryan Bischof, head of AI at Theory Ventures. Prakash asked about the firm's twenty twenty six builder survey. Bryan expected open source models to gain ground in production. Instead, the balance shifted toward closed models. [00:02:39] S4: There is a lot of, I would say, acceleration in model drops. And so right now, tying your production service to open source models requires a little bit more thrash in the engineering. And so I heard from a couple folks specifically, yes, but the open source models change so fast, it's a little bit more convenient to pin to an OpenAI model.
00:02:39 · Bryan Bischof, Nathan Labenz, Prakash Narayanan
[00:02:39] S4: There is a lot of, I would say, acceleration in model drops. And so right now, tying your production service to open source models requires a little bit more thrash in the engineering. And so I heard from a couple folks specifically, yes, but the open source models change so fast, it's a little bit more convenient to pin to an OpenAI model. [00:03:08] S0: This is the AI in the AM weekly highlights with my cohost, Prakash Narayanan. I'm Nathan Labenz. The narration is my cloned voice, and we'd love your feedback on what works and what doesn't. Part one, imminent means now. Friday's show opened on two documents, the METR report and an amendment to the Anthropic s one, the filing for its IPO. The new language, Anthropic says it may imminently meet its own threshold for dramatically accelerated progress attributable to AI. [00:03:46] S1: Anthropic did an update to their s one, the IPO filing. And in that update, they added language saying we are imminently about to see recursive self improvement. I've always had in my mind why they're doing the IPO and why so quickly. And my take has always been, I think within the company, there is a sense that you need more controls. More people have to have exposure and more control over this. And one way of getting this out there is getting the IPO done so that you give a larger part of society hooks into it and the ability to kind of express their desires through the market, etcetera, etcetera. To me, imminent is basically, like, you know, of... Like a month.
00:04:31 · Nathan Labenz
[00:04:31] S0: Usually, what I assume when I hear these sorts of statements is that it means now. The best model I have of Sam Altman and OpenAI leadership, which I formed during the six months between when I started testing GPT four and when it was finally launched, and watching very closely all their statements during that time to try to figure out how to read their statements in general given the inside information that I had then. The best way I can interpret them is they are speaking very literally and dressing it up as, like, vagary or, like, layering on uncertainty to try to, I don't know what exactly, keep the secrets a little bit longer while telegraphing the main message. It's sort of a strange strategy to me, honestly. I really don't know why they do that. And so I would interpret this, you know, probably the same way from Anthropic. And I think you could... I mean, it's really not hard to see how this would happen. Right? I mean, we have so many results now with all these Galaxy Brains zero day exploits. OpenAI just dropped some huge number of vulnerabilities in the Linux kernel. I think, like A thousand. A thousand of them. You know? So so much for many eyes and many hands make light work. Right? I mean, we... The idea, of course, with open source software over time is that because there's so many eyes on it, vulnerabilities will not last. They'll be spotted. They'll be patched. That was the whole promise and virtue of open source software. And it basically worked largely while the attackers were all human. It's almost certainly the case by analogy to Linux and by analogy to math that there is a lot of low hanging fruit, and a lot of low hanging fruit means acceleration. It means you're gonna get more value from compute, means you're gonna find more ways to get value from data. A big part of the reason I think they're sharing so much of this stuff is kind of to signal that, like, the same thing's about to happen on the ML stack. It seems pretty clear that this is now a a really credible threat. And, what are we gonna do about it? It becomes, I think, a sociopolitical question as much as a technical question. My... In my mind, the technical question is, like, mostly answered that there's gonna be a period of pretty dramatic fast progress. Does that top out on an s curve? Like, my my guess is still probably yes.
00:06:57 · Nathan Labenz
You know, I don't think it goes exponential forever, but who knows? And as, I think it was Rosie Campbell said, the s curve can stay steep longer than you can stay relevant. So that's kind of my expectation for the next few months is that we pick a ton of low hanging fruit. The models pick a ton of low hanging fruit in the stacks used to train them. As a result, we do see really fast progress in the short term. And then maybe that levels off, but it probably levels off where we have AIR and D at kind of a weekly superhuman level. And how that plays out over the course of 2027 is, like, really, really hard to predict.
00:07:36 · Prakash Narayanan
[00:07:36] S1: I agree that I I think imminent means now. I was also sandbagging with a few weeks to a month. What it means is that they have the tools available that they can see a path to implementing RSI pretty much anytime that they want. And that a lot of the pieces have been built, but they haven't pushed the button yet. They waited for the election, they're going to do the IPO, then they're going to announce of the new things that they've discovered. You're to have this gap up. And then we're going to have the Christmas break. And then during that Christmas break, we're all going to digest all of this new technology that we have access to. And then we're going to come back in January, and then we're going to see the takeoff as people deploy. Which is somewhat similar to what happened end of last year. So... Well, here we are, early early takeoff. So, you know, it's... I'm I'm somewhat wistful, but, you know, nostalgic for the human era. Is the end of our dominance at least. But hey, I've also made my peace with intelligence in the world makes the world better for me. If I had people of Apple's intelligence, people who designed the iPhone design like my bathroom or cars or home, I would be happier. That infused intelligence in all the products that you see makes your life better. So yes, it's gonna make everyone's lives better, but, yeah, it is it is the end of the Anthrocentric era. Mhmm.