Latent Space: The AI Engineer Podcast
00:38:10 · Shawn Wang, Jamin Ball
[00:38:10] S1: He's been on the pod... Cooking pod. Oh, nice. Nice. We made some soup for researchers. [00:38:15] S0: Amazing. Amazing. That's... I I love that pod. But, you know, when they saw Navier Stokes, that big math problem, you know, Mark came out and said, hey. Here's a model. It it may be dark blue. Light blue is harder to see, but, like, the top bar is an internal model we used to solve that problem. The blue bar is Astra. Hey. Look at how much more powerful the internal model is to what's publicly out there, which on some ways that like could scare you a little bit. Right? Like, hey. These labs have even more powerful models. So if you think about what's the gap between open weight and Frontier today, I think it's actually far wider than people expect because there are just far better models at the Frontier that haven't been publicly released yet, either whether that's safety, testing, alignment, some combination of everything, you know, unclear. But I do think you're still seeing the frontier accelerate even if it may appear that what's released is plateauing. I'm not even sure if it it does look like what's released has been plateauing. [00:39:16] S1: No. No. No. No one's saying it is plateauing today. It's it's it's... We're all discussing could it plateau in the future. Yeah. Right? Because that that is a pretty important thing to figure out [00:39:25] S0: Mhmm.
00:39:25 · Jamin Ball, Shawn Wang
[00:39:25] S0: Mhmm. [00:39:25] S1: For your investing Yeah. Which is part of the the deal here. Totally. Yeah. So so that is a very well phrased point. I thought you were going a different area where you're gonna argue about bitter lesson and you didn't. So kudos on that. And, like, I I think that's a very, very fair point that I I think we're, we've got still every generation... Model generation, it still shows quantum leaves. Mhmm. This is only in math. Mhmm. But it only has a pretty good track record of generalizing beyond just math Yeah. Or or coding. Yeah. Right? Totally. And so I I I do think that is. My point was a little bit more as on the financial analyst hat on Yeah. Which is you had this one time seven x improvement in revenue efficiency or whatever. And, like, it's not like they raised prices. Yeah. Yeah. Right? So like, where is it coming from? Oh, here's another good one. They didn't raise prices. [00:40:14] S0: Here... Oh, I think I have a chart. [00:40:15] S1: So you know what I'm saying? Right? Yeah. They have a 100... They have 100
00:40:15 · Shawn Wang, Jamin Ball
[00:40:15] S1: So you know what I'm saying? Right? Yeah. They have a 100... They have 100 [00:40:19] S0: Yeah. It's actually opposite. This is an awesome chart. Yes. This shows the cost to compete They're charging less. Yes. Exactly. The the cost They're charging less and making more money? The cost to complete a task with each model family. And so what you see is with whether it's Fable or Opus, and then you have Fable five one and Opus 5.5, you see about a 25% to 40% reduction in cost to complete the same task with each successive model family. And so it's kind of like, yes, the costs are going the costs are going way down, Their compute efficiency is going way up. And I think a lot of that just demonstrates, I mean, again, it's gonna sound a little like VC cliche, but like Jevons paradox. Oh my know. I know. [00:41:03] S1: I know. You can't have [00:41:05] S0: a podcast on AI without that coming [00:41:07] S1: up. Yeah. Know. It is... This is the appropriate time to bring it up. Yeah. Yes. [00:41:10] S0: And as as things get... You know, you go back to like the early days of the Internet. Like, many bytes were traveling across the Internet in 2000? How many bytes are traveling across the Internet today? Like many, many, many, many orders of magnitude more. And guess what? The price to transfer bytes over the network has dropped many... I mean, anyway. [00:41:29] S1: It's a switch meter. [00:41:30] S0: It's and Q. Price has gone way down. [00:41:34] S1: Yeah.
00:41:34 · Shawn Wang, Jamin Ball
[00:41:34] S1: Yeah. [00:41:34] S0: Quantity has gone way up. Yeah. And I think you're gonna see the same thing with people buying and serving models and tokens. [00:41:42] S1: Yeah. So so generally pro support of that, I I do want to remind people that that is a true objective fact. This is not, like, VC bullshit. Like, this is, like, actually how the the businesses are running. Okay. And it it is surprising to us Yeah. That that I get while... Yeah. We we read about this in theory, but, like, to actually see this in financials is, like, really ridiculous. [00:42:03] S0: You'll continue to see it. Think about JEV and decision models. And the price point of, Hey, instead of you want a binary classifier, you don't need a language model to do that. Just go use a decision model. Super cheap, super efficient. I think you're gonna continue to see efficiencies like that. You know, I go back to S3 and EC2, AWS's big initial products, storage and compute. You've seen 99% price reductions in both of those services from the day they were launched. But like what's happened to the EC2 and S3 business or just the broader storage and compute, more CPU compute, it's gone up exponentially. I mean, all the hyperscalers reported today, they're all over a $100,000,000,000 run rate businesses today, if they were individual companies. Google Cloud's grown over a 100%. It was just crazy. Azure's growing almost 50. Amazon reported today, we're recording on Thursday, 39, almost 40%. Like, these businesses are just accelerating and growing at It's crazy.
00:42:59 · Shawn Wang, Jamin Ball
[00:42:59] S1: East mode. It's crazy. You should drop everything you're doing and just go start one of these Yeah. Yeah. Yeah. It's October 29 for those those keeping track. And and, yes, a fun personal anecdote. I was in my hedge fund when... The day they broke out am... AWS from Amazon. No kidding. And at the time, we were not long Amazon because we were worried about AWS margins. Yeah. And also, like, oh, is it just startups? And what if, like, you know, we go through a startup winter and, like, all that? Yeah. It was so nonsensical. So, like, it it... The the funny thing about finance is that you can do all these works and make worse decisions. Yeah. Yeah. Yeah. We definitely did that. [00:43:36] S0: Totally. Totally. I mean, that... That's the thing. Like, market timing's impossible. Like [00:43:42] S1: And also fixating on the wrong thing Yeah. Which a lot of people are are having to encounter for the first time. That's right. You have 2,000 data points that were released Mhmm. In this thing. Mhmm. And you will fixate on one thing, and it'll be like, that is that is the everything you need to know. And, like, well, no. Like, you you lost sight of the bigger picture.
00:43:42 · Shawn Wang, Jamin Ball
[00:43:42] S1: And also fixating on the wrong thing Yeah. Which a lot of people are are having to encounter for the first time. That's right. You have 2,000 data points that were released Mhmm. In this thing. Mhmm. And you will fixate on one thing, and it'll be like, that is that is the everything you need to know. And, like, well, no. Like, you you lost sight of the bigger picture. [00:43:57] S0: Yeah. I mean, that's that's for me. I mean, again, there's a lot of things that I believe that others would disagree with me on. Yeah. But I believe there's one question that matters, and that's can Anthropic stay at the frontier? Yeah. And to me, if I believe the answer to that question is yes, and I think everything they've done historically has demonstrated that, I think they'll build a great business and will continue to. [00:44:14] S1: But... Okay. But then, like, there's a right price for that.