"The Cognitive Revolution"
00:20:48 · Brian Zhou
[00:20:48] S3: Yeah. So the overall sort of result that we're seeing right now is that in general, models perform just as well as the best forecasters in terms of humans. In the House and in the Senate, agents perform slightly worse than the markets but better than the modelers. And in the Senate, the best agents like GPT six Astra and CloudFable five point one perform under the best forecaster, Nate Silver, but ahead of the markets. But all of these are sort of within this, like, very close margin of error. And it's important to note that a lot of the results for very competitive races have not necessarily come out yet. Right now, there's still around 100 to 200 races that across the House, across state governorships, there are still two races in the Senate that have yet to be called. So these are very preliminary results. It definitely is important to note though that for a lot of these agents, various methods that they have include aggregating, for example, Nate Silver's decision model or aggregating prediction markets that they see overall. So it's definitely not as if they are suddenly just spitting out a number that is not based off of any of the numbers that they're seeing. But we also find that these models are very well calibrated. For example, if a model assigns an 80% chance of a candidate winning across all of the elections where assigns a similar 80% chance, it is right around 80% of the time, which is essentially what you do to measure the calibration of these models. And in this, we find that the best models are quite well calibrated. We're more excited about the results of what this means for later models as well because unlike experts or unlike prediction markets, for example, there's only going to be a thousand dollars of spread, for example, on what we might see in, like, a very local race down the ballot. Or expert modelers may not be forecasting what individual state senate seats or state house seats are going to contain. So this is somewhere in which agents shine simply because modelers don't have the attention, and there's not enough people betting on these much smaller markets, essentially.
00:22:47 · Nathan Labenz, Brian Zhou
[00:22:47] S0: Was gonna ask about the the relationship between prediction and trading. I saw a note that indicated that Fable five one had the highest Brier score on its predictions, but still didn't do so well in the markets, and that sounds like the kind of thing we might get something from. [00:23:05] S3: Yeah. So part of the report will also sort of cover this essentially multi agent behavior. Right? Because we essentially have one agent working as the forecaster and one agent working as the trader. We set up infrastructure that allowed them to communicate with each other. So we're seeing a lot of these instances of, for example, models making up information, parsing information wrong, and these sorts of like other broader issues, which partly explains why traders... The actual trading agents didn't do too great. [00:23:33] S0: Thoughtful also leads post train bench. Give an agent a base model, a GPU, and a time limit, and see how well it can post train the model. With new results still to come, I asked Joe what he expected next in light of the METR report. What what is your expectation as somebody who's, like, spent serious time looking at what models can do to train other models? Like, can you put a little more color or sort of, you know, nuance onto your expectations for the next couple of quarters?
00:24:08 · Brian Zhou
[00:24:08] S3: Yeah. I mean, I think one of the general trends that we've been seeing both with the report from yesterday and with earlier reports, whether this was Anthropic's measurement of how well individual researchers have been uplifted by AI, whether looking at metrics like lines of code written, experiments done, is the general trend we're kind of seeing is that... I mean, today, and this has already been true for many months, researchers have essentially agents that can do the vast majority of the actual implementation work, which is what we would kind of consider as, like, research engineering. So the realistic limitations that researchers today have is not necessarily the, like, ability to execute on any of their ideas, but as, like, many people have talked about, right, their sort of research taste in finding what problems and experiments that they want to do, as well as just the, like, speed at which they're able to come up with these, right, given that agents are already autonomously building and executing on a lot of the experiments that they want to run. So I think regardless of whether we get to recursive improvement, we have definitely already seen a huge uplift that will continue to only get better. And then I think it really just becomes the sort of end bottleneck just being resources and quality of ideas and intuition, right, which is, of course, the next hill that everyone is trying to climb.
00:25:28 · Nathan Labenz, Brian Zhou
[00:25:28] S0: Have you seen anything that you would say starts to look like the distribution of idea quality is overlapping between the models and truly top researchers? [00:25:39] S3: I don't think so necessarily, but I do think we have observed that with better models that currently exist right now, for example, they exhibit a larger willingness to fan out, do experiments, a better both ability to think of, like, what might be interesting experiments and execute on those experiments. Like, we definitely do see with worse models, more of, a general, like, linear approach to whether it's post training or other things. And then with more and more capable models that we observe and test, we do see this general behavior of, oh, it's, like, spending its first few hours trying to like fan out, do a bunch of different experiments, allocating its resources in a pretty intuitive way and doing what we might be thinking of. None of this is necessarily a Move 37 moment because a lot of this is just the agent thinking, oh, like, let me try an initial SFT batch and see how that affects the results, for example. Right? So we don't necessarily see anything that would be indicative of superhuman taste or anything else. But we definitely do see that more capable models are just better at long horizon work and better at the sort of autonomy that we would expect we would want with better researchers.
00:25:39 · Brian Zhou, Nathan Labenz
[00:25:39] S3: I don't think so necessarily, but I do think we have observed that with better models that currently exist right now, for example, they exhibit a larger willingness to fan out, do experiments, a better both ability to think of, like, what might be interesting experiments and execute on those experiments. Like, we definitely do see with worse models, more of, a general, like, linear approach to whether it's post training or other things. And then with more and more capable models that we observe and test, we do see this general behavior of, oh, it's, like, spending its first few hours trying to like fan out, do a bunch of different experiments, allocating its resources in a pretty intuitive way and doing what we might be thinking of. None of this is necessarily a Move 37 moment because a lot of this is just the agent thinking, oh, like, let me try an initial SFT batch and see how that affects the results, for example. Right? So we don't necessarily see anything that would be indicative of superhuman taste or anything else. But we definitely do see that more capable models are just better at long horizon work and better at the sort of autonomy that we would expect we would want with better researchers. [00:26:53] S0: Great to meet you. We'll be looking out for the next version of post train bench, and you've got me wondering what the agents will say about how much acceleration we're gonna see in AI over the next couple quarters. If they can predict the election as well as the experts, we should probably start asking them about the future of AI itself.
00:26:53 · Nathan Labenz, S5
[00:26:53] S0: Great to meet you. We'll be looking out for the next version of post train bench, and you've got me wondering what the agents will say about how much acceleration we're gonna see in AI over the next couple quarters. If they can predict the election as well as the experts, we should probably start asking them about the future of AI itself. Ad[00:27:14] S5: Hey. We'll continue our interview in a moment after a word from our sponsors.