No Priors: Artificial Intelligence | Technology | Startups
00:17:14 · Noam Brown
[00:17:14] S0: I think absolutely. I think actually a really great example is the Erdos unit distance problem. So for the viewers that don't know, like, we used an internal model at OpenAI a few weeks ago to disprove the unit Erdos unit distance conjecture. Now I'm not a mathematician, but this seems like it was a a pretty big deal. The in the math community, it was, like, the first first problem that a lot of mathematicians had really spent a lot of time on, and the model was able to do something that they weren't able to do and do it in a way that was actually interesting and useful for mathematicians. Honestly, it did it at a budget that was dirt cheap. I mean, we didn't put a lot of effort into this. We just we trained a new model, and we were just curious what it could do, and we ran it through some problems. And this one, at a pretty low budget, it was like, oh, yeah. I think I have a disproof, and then we were able to verify that, yeah, that disproof is correct. After we announced the results, a bunch of people found that you could get the answer out of 5.5 as well. If now it's not as simple as just asking 5.5, hey. Here's the neural unit distance conjecture. What's the disprove? You had to scaffold it a bit. You had to, like, steer it a bit. And so somebody found, okay. You asked 5.5. List a bunch of ways that you could tackle this problem. And then for it it lists one of the paths that are actually promising to get to the disprove. And then it you tell it, like, okay. Explore this some more. And then if you do this enough times, it actually ends up arriving at the disprove. Now what this means is you could, in principle, ask 5.5 to you know, as as a general purpose scaffold, list a bunch of different strategies, and then for each strategy, tell it to investigate that strategy. And then it would probably be able to arrive at the disprove with a general purpose scaffold. Now that scaffold would be very expensive. I mean, it would probably cost I I just ballpark, like, a thousand to 10 to a $100,000. But it would be possible, and it would have been possible for somebody to disprove the Erdosia to disc distance conjecture before we did using a general purpose model. And nobody had explored sufficiently, well, happens if I put a $100,000 worth of compute into 5.5? What could it do?
00:19:17 · Noam Brown, Sarah Guo
And the answer is, yeah, you probably could get stuff like that out of it. [00:19:21] S1: So people should be experimenting more with the current generation in terms of [00:19:24] S0: Well, this is, I think, an interesting question of, is it worth it to experiment with because, again, the model release cycle is every every couple months, we put out a new model that's even more powerful. And so the cost of disproving the air to ocean at distance congestion drops by, like, 10 or a 100 x with every model release cycle, probably, in some cases, more. [00:19:42] S1: So You've seen the meme that's like, oh, alright. Like, why why bother doing any engineering work when I should just wait for the next [00:19:48] S0: model release? On vacation and come back two months later, and then it's, you know, a thousand times cheaper. So agree with that? I Is that [00:19:55] S1: what you're doing right now at OpenAI, just waiting for the next model release?
00:19:55 · Sarah Guo, Noam Brown
[00:19:55] S1: what you're doing right now at OpenAI, just waiting for the next model release? [00:19:58] S0: I think I I mean, I will say we're in we're in a period where progress is very fast, and, like, yeah, the models are becoming more capable. I I can say, like, at OpenAI, one of the things that we're we're actively not doing look. We have a lot of mathematicians. We have a lot of physicists. People are very excited about what these models can do right now Mhmm. Especially, you know, the internal models. We are trying to encourage people to not spend all their time just, like, going through all the mathematical open problems, physics problems, and just seeing pushing the models to their limits to see what they can prove or disprove Because we really think the focus should be on how do we make even more capable models, how can we get them get them out safely to the world as quickly as possible so that all the scientists in the world can use these models to solve the problems themselves. So, yeah, in some sense, we are thinking about this that, yes, it's really tempting to just put all of our efforts into scaling up these models and see what they can do with their limits right now, but really the focus should be on how do we use these models to make even more powerful models, even more capable models that can do everything much more cost effectively.
00:20:59 · Sarah Guo, Noam Brown
[00:20:59] S1: What is changing about the direction or allocation of resources for research in your mind given your beliefs about this very large scale the impact of very large scale test time compute? How does this interact with the idea of recursive self improvement, for example, where, you know, it's a dominant idea for how, you know, any lab gets to the best capability model. [00:21:23] S0: So one thing I should clarify, I don't think we're at the point where, okay, you just give it an arbitrary an an extremely high inference budget, and it's just it's just super intelligent across the board. [00:21:32] S1: Slash goal [00:21:33] S0: Yeah. Baseline. Okay. G p d seven or whatever and then, like, yeah, just go nuts. [00:21:37] S1: What's between us and there then?
00:21:39 · Noam Brown
[00:21:39] S0: I think having played around with the model so okay. So first of all, there are some benchmarks where the models will just not improve if they have more inference budget. So I think a lot of factual retrieval kind of questions fall into this category of if you ask a person when was Abraham Lincoln born and they don't know the date, they could sit there. They could think about it for a week. If they if they don't have access to a computer or something, they're not gonna be able to do better answering that question if they thought about it for a week compared to five seconds. Same with the model. If you actually, interestingly enough, if you give the model these kinds of, like, factual retrieval questions and you give them a little bit of time to think, they do actually do But if you give them a week, they're not suddenly gonna do better at remembering dates. There are so there's some benchmarks where they clearly improve with more test time compute, and there's some where they don't. I think on the other extreme, there are benchmarks where they kind of obviously will keep improving limit without limit with more test time compute. So the example I like to point to is sudoku. If you it's there's a really simple strategy to solving sudoku, which is just try a bunch of different random numbers and then see if it fits the criteria, if it if it matches all the constraints. And if it doesn't, just try a different random combination of numbers. And, clearly, with enough time, you will be able to solve any Sudoku puzzle with this strategy. You can kind of trivially see, like, okay, any model could keep doing better and better if it was just given more test compute. So you have and and all the benchmarks kind of exist somewhere between these two extremes. The models are not at the level where if you just give them enough test time compute, they will be able to do all of our jobs just because, yeah, there's some benchmarks where they will not improve. There are some things where they they will not improve. One thing I see for research in particular is they don't have very good research taste right now, and so I think they're actually a very good complement to researchers, especially, you know, I've found, like, I've found that I've been much more effective by using these models, but they're not able to fully replace the whole research cycle.
00:23:38 · Noam Brown, Sarah Guo
Now does that change with time? Probably. I mean, I think the models are getting better across the board. Some some things are getting better faster than others, but they're not at the point where they're fully replacing researchers with just enough test time compute. [00:23:51] S1: Can you give an example or two of, like, asking the model to do a research task for just like this? Is a terrible idea. [00:23:59] S0: I mean, I I think going back to my poker solver example, I was really impressed with the model's ability to optimize the algorithms that I had developed in my in my PhD. It was honestly it it was it was shocking to see how inefficient I was, in retrospect, and they were able to make it, like, you know, 1,000 x faster. And then I was like, okay. Can you come up with an algorithm that is better than the algorithms that I came up with or that anybody else came up with? You and go ahead and, like, look at all the published work and synthesize that and then try to come up with something novel, and it it's not able to do it. And I can give it a lot of time, it's it's still not able to do it. Now it's possible that if I scaffolded something and, like, kind of constrained it a bit more, that maybe it could eventually come up with something better, but it would take a lot of it's not just as simple as saying, okay. Please come up with a better algorithm. [00:24:52] S1: And how do you think that that gets improved?