00:16:35 · Amit Jain, Finn Meeks
[00:16:35] S0: Sorry. I don't understand. [00:16:36] S1: Like, did you have image data, audio data involved in in those as well?
00:16:44 · Amit Jain
[00:16:44] S0: So image data, yes. Audio tower is not there today. Right? So in order to make it a true world model, you you need to have like, you know, a bit more signals. So video is... Video and audio will be the next one basically. And I think once you have video audio language, you have about... You're about 90% of the way there. You have pretty much everything. I mean, some of you who are in robotics here would make the case you don't have grasping information and all these kind of things. Right? And it's true. You don't have a smell. You... That's true. I don't know if robots need smell, a sense of smell. They can just have sensors for some critical things like LPG leak, and those kinds of things, carbon monoxide detection, those kinds of things. And I think it will be sufficient. Maybe not in the fullness of time, but initially. But yeah, you need audio, you need video, and you need language. I think those are really, really important. And then everything else that comes after it is helpful. Because even if you don't have the exact sense of like how much grip strength you should have, you do know from observing videos that like, hey, something like this can be picked up. But if you see like an elephant, oh, let's not pick it up. It doesn't matter how much strength I put it into that. I know my own strength and I kinda understand that. Then you do on policy training in that bold model sense of doing actual pickups, right? Will it crush a lot of cans initially? Yes, but that's okay. You learn very quickly. They're like, oh, this is not the pressure I apply here. This is the pressure... This is how humans learn as well. So human brain generally bootstraps from like all of these kind of things, and then you go forward with by learning the physical world and length of your arm and the strength of your arm and all these kind of things as babies try to grasp. So that's how we think it will go.
00:18:18 · Finn Meeks, Amit Jain
[00:18:18] S1: With this unified model, do you do you think that this eventually replaces language models as well? Yeah, I'm curious, like, is LMs just going to be a thing of the past a few years from now and everything will be structured more as unified models? [00:18:34] S0: It depends on what you mean by that. If you mean that empirically, I think no. If you mean that theoretically, I think yes. So empirically, the considerations are different. When I say empirically, I mean in the field, in deployment, in real usage. In real world usage, you want to strike for efficiency a lot more. So I think LM scaling is now at a point where you just don't see like, oh, okay. So for instance, the recent Qwen 27B model actually is outperforming a Qwen 289B model basically, which is ridiculous at this point. I think we will still continue to use language models quite a bit, but theoretically, a world model which has understanding of language and physics and the world model will strictly be better. It will be more general, it will be able to use in more scenarios, But does that mean you should be using that role model for writing front end code? Maybe not. But if you're making a front end that is really involved and has a lot of visual states and a lot of interactions on it, then yes, you do want to have the world model that can understand the full layout and flow of it and can like, you know, make makes and watch the video of the thing play out and say like, no, no, this is not very good. Like, you know, let me edit on it. So use case.
00:18:34 · Amit Jain, Finn Meeks
[00:18:34] S0: It depends on what you mean by that. If you mean that empirically, I think no. If you mean that theoretically, I think yes. So empirically, the considerations are different. When I say empirically, I mean in the field, in deployment, in real usage. In real world usage, you want to strike for efficiency a lot more. So I think LM scaling is now at a point where you just don't see like, oh, okay. So for instance, the recent Qwen 27B model actually is outperforming a Qwen 289B model basically, which is ridiculous at this point. I think we will still continue to use language models quite a bit, but theoretically, a world model which has understanding of language and physics and the world model will strictly be better. It will be more general, it will be able to use in more scenarios, But does that mean you should be using that role model for writing front end code? Maybe not. But if you're making a front end that is really involved and has a lot of visual states and a lot of interactions on it, then yes, you do want to have the world model that can understand the full layout and flow of it and can like, you know, make makes and watch the video of the thing play out and say like, no, no, this is not very good. Like, you know, let me edit on it. So use case. [00:19:47] S1: So you might see kind of just systems of models, with a unified model guiding guiding them tailored to a specific application.
00:19:47 · Finn Meeks, Amit Jain
[00:19:47] S1: So you might see kind of just systems of models, with a unified model guiding guiding them tailored to a specific application. [00:19:54] S0: Correct. The unified model will replace LLMs as the core, but you might do a tool call to an LLM and ask it like, you know, hey, can you just code this out because you're more efficient than me? [00:20:03] S1: Tell us a little bit more about Luma Agents. This is a recent release of yours, and I I think it kinda goes back to to what you were saying earlier is that there's a lot of value too in just the the orchestration and the ability to actually, like, solve, you know, that end to end problem for our customer beyond just having a good model. And there's more than just a model that goes into it. Can you tell us a little bit more about Luma Agents and just how you actually think about making these useful for customers?
00:20:27 · Amit Jain
[00:20:27] S0: Right. So Luma now has a very fast growing and sprawling business in entertainment, media, advertising. Basically the use cases for visual models when the accuracy is good enough, where humans think it looks real, but not good enough yet where it's metrically accurate. So for these kind of works, it's actually really useful. Just to give you a sense of that market, by the way, on average, companies spend about 10% of their revenue, nonprofits, revenue on telling their own stories. Every company, if you're a consumer, you're going spend about 15% to 20% of your revenue on telling the world about it. If you are a services company, you're going be somewhere in the 10% to 12% range. And if you are a government or defense or those kind of companies, you're going to spend less, not zero, less. How many of you know about Palantir, right? Like how many of know about Northrop Grumman? You have no business knowing about Northrop Grumman because none of us are going buying jets like on Sundays at IKEA. But we know about Northrop Grumman because well, at the end of the day, the dollars come from our funds and then from our like tax, right? So every company needs to be known in the world. So that's a very big business. So if you have a stock market that is trading at 30,000,000,000,000, right? About 3,000,000,000,000 of that value is actually going towards telling that story, media advertising. But the thing is like, and this doesn't apply just to our business, it applies to everything. Promise of AI is not doing a little bit of spot work, all right? Hey, I can make copy. It's like, all right, good for you, man. Can you actually make the book for me fully? Can you actually just go talk to the customer? Hey, I can make a little bit of image for you or video for you. It's like, who cares? Right? Can you actually do the full end to end movie or can you do a campaign? In 2024, 2025, we struggled with this a lot because agents weren't really there and the lens weren't really good enough for that. Our models were also really not good enough to do long horizon tasks. But we just actually announced a full production on Prime Video with Sir Ben Kingsley. It's a TV show where the production budget is $4,000,000 an episode and it has Sir Ben Kingsley, it's called Moses, right? And now we are at a place where it is about like 90% produced with Luma Agents.