Going Direct Conversations
00:15:25 · Chris Paxton, Michael Cho, Jordan Wolfe
[00:15:25] S0: Everybody's house is different. [00:15:27] S1: Yeah, exactly. In that paper, they have this success rate. You just have to take it that, okay, they did their work and it is what they say, but no one can verify it. [00:15:38] S0: Sure. It's getting better, but yeah, definitely not. Oh, that's so hard. Yeah. [00:15:42] S2: All right. Well, it's an interesting question, right? I guess this one's for you, Chris, because obviously you're working on or agility is working on the humanoid form factor. And it's kind of like when the world of robotics is like, just show me. It's very different than some abstract or benchmarking process to benchmark LLMs. [00:16:07] S0: Is that okay? Yeah. The thing is that you can't just get a good feel for how a robot works from a video because you can always get a good video. Honestly, the other thing is that I think people have seen lots of really good robot videos for a long time because they've seen Star Wars or whatever. You've seen robots that can do things that they can't even do now. I think you have to be able to show robots in the real world and see it for yourself to be able to see what it can do. I think a lot of videos can be misleading. I wanted to drill into the different types of training, methods for training.
00:16:07 · Chris Paxton, Jordan Wolfe
[00:16:07] S0: Is that okay? Yeah. The thing is that you can't just get a good feel for how a robot works from a video because you can always get a good video. Honestly, the other thing is that I think people have seen lots of really good robot videos for a long time because they've seen Star Wars or whatever. You've seen robots that can do things that they can't even do now. I think you have to be able to show robots in the real world and see it for yourself to be able to see what it can do. I think a lot of videos can be misleading. I wanted to drill into the different types of training, methods for training. [00:16:44] S2: I wanted to get your perspective on it. Because that's also another aspect that's kind of a little bit unclear. What is the most effective? Is it a mix of all of these? Let's start with just teleoperation. Michael, what's your perspective on what does teleoperation truly allow you to do to solve some of the training gaps that currently exist?
00:16:44 · Jordan Wolfe, Michael Cho
[00:16:44] S2: I wanted to get your perspective on it. Because that's also another aspect that's kind of a little bit unclear. What is the most effective? Is it a mix of all of these? Let's start with just teleoperation. Michael, what's your perspective on what does teleoperation truly allow you to do to solve some of the training gaps that currently exist? [00:17:08] S1: Yeah, I would say teleoperation is the gold standard amongst the three types of data I mentioned just now. But obviously, it's extremely costly to collect. I say it's a gold standard because you are doing in the real physical world with a real physical robot. So you don't suffer from, let's say, embodiment gap. And you collect exactly, typically, I mean, this is humans controlling the robot, right? So you also, in a way, collect human intention, like what would a human do, like in this instance, right? So there's some social compliance stuff that also naturally get collected. So I think it's great for all those reasons. But of course, these robots break all the time, you know, and you can't respawn them, you know, like in a simulation. So that's a huge thing. Yeah. So I, I think my thing is that if you can gamify it, then, then hopefully you get more people to constantly do it, but you still cannot get away from the fact that these things will break all the time and makes it very, very hard to scale. [00:18:08] S2: Yeah. Yeah. What's your thought, Chris, on the teleoperation side?
00:18:08 · Jordan Wolfe, Chris Paxton
[00:18:08] S2: Yeah. Yeah. What's your thought, Chris, on the teleoperation side? [00:18:13] S0: I mean, I think that, yeah, like you're saying, that's the gold standard, getting real robot data. And I think the gold standard is actually probably, the best data is probably the data the robot collects itself. It's probably like reinforcement learning kind of data. But that's pretty rare still. But the robots are... Teleoperation data is great as long as you can actually get it, right? But while scaling, it's hard. And so that was actually the next question was kind of reinforcement learning. [00:18:45] S2: So Chris, can you kind of explain that for folks that might not be so understanding of what that actually means in the term of robotics space?
00:18:53 · Chris Paxton
[00:18:53] S0: Yeah, sure. Yeah, so I apparently even wrote a book chapter on this for what it's worth. So reinforcement learning is... What's the best way to describe this? I really should have a better answer for you off the top of my head. But it's basically the idea that the robot needs to be able to teach itself from its own experiences. So the way that you or I learn to perform a task is not by watching. I don't watch... If I wanted to learn how to... hit a tennis ball. I think that's like to serve a tennis ball or something like this. Like I wouldn't do that by just watching a bunch of YouTube videos. I would do that by tossing a tennis ball and trying to hit it, right? And then do practicing for days or weeks or months or years depending on how good I wanted to get, right? And that's what reinforcement learning is in general. It is saying that the robot is going to just like sit there and take in information about the world and it's going to And it's going to make a decision and try something, and if it fails, it's going to take that and think about it and do better next time, hopefully. Or if it succeeds, it's going to be like, okay, that was good. Let's try again a little bit. Let's try to do more like that. And the reason that I'm saying it's the best is because it's the same reason that it's best for you or me. If you want to learn to do something, You'll have your own embodiment and your own version, your own thoughts about how to do that, and you're never going to really get that from watching someone. But you'll do pretty good from watching someone. You'll learn a lot from watching someone, but you still at some point always need to do it yourself.
00:20:27 · Jordan Wolfe, Chris Paxton
[00:20:27] S2: Right? And so that's a good segue into kind of another aspect, which is kind of one I really wanted to better understand, which is learning from kind of human video. So the concept of can you collect, you know, you collect data, you label it, you provide contacts on and on and on. You have your own data schema. So Mike, what's kind of your perspective on, because of just what Chris said around like you actually have to do the thing to learn the thing. At some point, at some point. [00:20:55] S0: You can get pretty far without it, right? Let's dig into that because this is like a very abstract thing for folks that [00:21:01] S2: aren't in the robotic space. So how useful is kind of human video for training robots, Mike? I mean, do you think, is that a critical piece right now? And will it be a critical piece going forward? Or is it going to be the robot physically doing like physically watching someone and not video.