00:48:27 · John Collison, Sam Altman
[00:48:27] S0: It feels like the industry is still figuring out the being nice to work with, where there was the glazing era. And more recently, there was the stroppy, ornery era of models that were anti glazing, and as a result, tried to disagree with you and nitpick at you on every minor thing. The current ones are pretty good, actually. [00:48:44] S1: It's gotten... I think the latest models are Yeah. Much better than we've had in a long time. [00:48:49] S0: But it also feels like there's probably higher peaks to Ascend. [00:48:52] S1: Emmett Shear said to me years ago that what he wanted was a really thoughtful and challenging but still respectful senior coworker. And I have still not heard a better, like, brief encapsulation than that. [00:49:06] S0: Yes. After working with him for many years, what's your grand unified theory of Greg Brockman? [00:49:10] S1: Think Greg Brockman is, like, one of a kind in the best sense. [00:49:13] S0: He's singular on the Greg Brockman scale, and you just kinda have to be there to understand what that is. [00:49:16] S1: But I think it's it's, like, a combination of, like, yeah, intensity, like, monster productivity, extreme dedication, and, like, focus on sixteenth of mission. [00:49:26] S0: It's it's [00:49:26] S1: very cool.
00:49:26 · Sam Altman, John Collison
[00:49:26] S1: very cool. [00:49:27] S0: Hate tokens as a metric for anything, really. Pricing or just kind of... It's not a standard unit of AI. And yet we found ourselves there. You know, we did talk... And you might say outcomes, but I don't think it can be outcomes. The outcomes is too Yeah. Different across different use cases. It's... What's the right standardized unit? [00:49:51] S1: We almost made a very... What would have been bad call to say we're not going to price our next model in tokens at all. And you're just going to give it a prompt, and we're going to give you a price back. And we're going to do it. And it's going to be like AWS. You just do something and we send you a bill. But the customer response on that is so negative. They're like, I need to know what I'm paying for somehow. Yes. And I asked a few people, like, would you rather pay more for the same output but have it priced in some number of consistent tokens or something, even if we inflated the tokens. And I go, I'd much rather inflate you to inflate the tokens and pay more, because I know there's a real thing there. So I think you are right that people don't just want us to give them a price because they feel like over the long term they're going to get screwed somehow, In the same way that maybe people would rather pay for CPU hours rather than just getting their thing done, even if it was less expensive.
00:49:51 · Sam Altman, John Collison
[00:49:51] S1: We almost made a very... What would have been bad call to say we're not going to price our next model in tokens at all. And you're just going to give it a prompt, and we're going to give you a price back. And we're going to do it. And it's going to be like AWS. You just do something and we send you a bill. But the customer response on that is so negative. They're like, I need to know what I'm paying for somehow. Yes. And I asked a few people, like, would you rather pay more for the same output but have it priced in some number of consistent tokens or something, even if we inflated the tokens. And I go, I'd much rather inflate you to inflate the tokens and pay more, because I know there's a real thing there. So I think you are right that people don't just want us to give them a price because they feel like over the long term they're going to get screwed somehow, In the same way that maybe people would rather pay for CPU hours rather than just getting their thing done, even if it was less expensive. [00:50:48] S0: You know the way, like, nanometer... The chip It's totally fine. Yeah. Yeah. The nanometers has very little relation to reality these days. Will we get something like that in tokens where you guys should start saying, here is an intelligence equivalent token like... I can't believe it's that butter. [00:51:03] S1: Of, the smallest feature? Yeah. I... That's, like, made up. Hope not. [00:51:08] S0: Think I'll do that. [00:51:08] S1: It... I mean, it offends me that we, like, use the word nanometers for something that's nothing new with nanometers.
00:51:08 · Sam Altman, John Collison
[00:51:08] S1: It... I mean, it offends me that we, like, use the word nanometers for something that's nothing new with nanometers. [00:51:15] S0: I know. It's very silly. Yeah. So... You're not like that. [00:51:18] S1: I I would rather not make up a fake unit. Tokens are a kind of fake unit. I mean, they're real in some sense, but they're not a good unit. [00:51:25] S0: They're fake from a business value point of view. But isn't the only... Okay. Isn't the only defense of tokens that if you're wasting tokens, we should fix that pretty soon? Like, we shouldn't be streaming... Claude Shannon would not approve of us just streaming noise rather than signal. And so there will be a... There's a floor of the information theoretic most useful tokens on both sides, and we'll come back to [00:51:49] S1: that later. Right now you have to think about whether you should want the 10¢ per million tokens out of a small model or $10 per token tokens out of a big model, and then whether... How long the chain of thought is likely to be. And that's just cognitive overhead that I don't think actually adds any value. [00:52:10] S0: Is the chain of thought included in the pricing? [00:52:12] S1: In what we charge. Most people do. [00:52:13] S0: Okay. [00:52:14] S1: By the way, the Guinness was not as bad as... Definitely not as good as an IPA, but not as bad. It's good. Yeah. I mean, an IPA is great, but it's [00:52:22] S0: it's pretty good. We like Guinness. Yeah. Yeah. At least in this podcast. Thank you.