00:27:21 · Adam Miller, Sachin
[00:27:21] S0: Wow. Sounds exciting. [00:27:23] S1: Something I'm very curious about is how a company from New York with kind of three founders outside of like Central Silicon Valley casting without as much money as, you know, some of the big kind of players in the space has been able to, you artificial analysis publishes these benchmarks and you guys are often at the top. How have you been able to do that in the kind of most simplest way?
00:27:47 · Anastasis Germanidis
[00:27:47] S2: Yeah. I mean, it's it's always sometimes the question I wanna ask the, you know, the big labs and big companies. Why is a smaller company able to too much, you know, to to build competitive models? I think there's a few a few ways to answer that question. I think the one of the most important thing is focus like video models and world models are the core main thing that Runway is focused on. We're not having a whole suite of products and like a lot of different kinds of models that we're working on. So getting this right is very important for us and it's the mission of the company that we succeed at this the second piece is that we've been doing this for a long time we released the first public available text to video model. Like we've learned through many iterations how how to build those models, like all the tricks that go into training them. A big focus for us has been this idea of data efficiency of essentially learning more from the same data of figuring out the water techniques that you can introduce in both pre training and post training to to get better performance with the same amount of compute. We have a team that's about 10x smaller and 100x probably roughly less compute and I think we've been able to be successful in being competitive with those larger players and I think a lot of it comes through with like the strength of the the research team and, like, all this accumulated knowledge and taste that has... For over years of, like, building those models.
00:29:21 · Adam Miller
[00:29:21] S0: So I think publicly, there's a little bit more information out there about how video foundation models are made. They've been made for a few years now, and the in vogue thing, world models, it feels like a little bit of a black box. So I'd love to hear from your perspective about how Runway is approaching world models in terms of research, whether there's potentially new form factors to utilise the technology and the interactions you're having with other industries?
00:29:46 · Anastasis Germanidis, Sachin
[00:29:46] S2: Yeah, so the most important thing to build a great world model for us is to build a great video model first. It's foundation that makes world modeling work is that the base video model needs to understand has a really good performance in physics and, in, that enables it to be, to be effective as a as a world model. World models, the way we build them is essentially as a post training phase on top of the base video model training that we do. As as we find all those applications of our world models in different industries it's very important to work very closely with the companies that have been doing this for a while. For example, kind of collaborations in robotics, of figuring out ways in which we can provide value to those companies in exchange for like the understanding how they operate, what are the biggest problems that they they need to solve. And, a big piece of also building, world models is getting kind of, a real time interaction to work. So, if you if you use if you use a video model, you typically see, you know, you wait for like a minute or two minutes to get back an output. With a world model, it needs to be a real time interaction, and getting to that kind of, low latency is a very difficult both research challenge and engineering challenge that that we [00:31:11] S1: need to solve. What's the most compelling use case you've seen your world models used for so far?
00:31:11 · Sachin, Anastasis Germanidis
[00:31:11] S1: need to solve. What's the most compelling use case you've seen your world models used for so far? [00:31:19] S2: The the most interesting use case that we didn't also expect to work that well in the beginning and ended up working kind of beyond our expectations was this idea of as a as a simulator for robotics. So the idea of being able to to test how your robotics model performs inside the world model and getting results that match what you would get in reality if you tested the robot on the same exact task. So we did a lot of work in establishing that correlation, that kind of simulation to reality correlation, where if a robot succeeds or fails inside the simulation, it succeeds or fails in the real world and there is good correlation between the two. Because once you get that correlation, then you can then evaluate your robotics models way faster. You can basically evaluate them, like, thousands or millions of scenarios before you even test them in the real world. [00:32:14] S1: Do you think there'll be application layer companies that sit on top of the world models, or do you think that people just use world models directly for these kinds of tasks?
00:32:14 · Sachin, Anastasis Germanidis, Adam Miller
[00:32:14] S1: Do you think there'll be application layer companies that sit on top of the world models, or do you think that people just use world models directly for these kinds of tasks? [00:32:22] S2: I think it's it's very likely there's gonna be a whole ecosystem around world models similar to what we've seen with, you know, regular image and video models, similar to what we've seen with language models. So there's gonna be more infrastructure companies, more, you know, aggregators. There's gonna be kind of harnesses on top of those models that help kind of combine combine them in different ways, so I think that's definitely going to be the case. [00:32:46] S0: And when you're trying to make simulations in world models, do you think these models, are they starting to understand the principles of physics or are they mimicking the physics of video that it has been trained on and seen before?