00:16:09 · Aditya Agarwal, Sergey Levine
[00:16:09] S1: Oh, that's super interesting. So is the is the model then actually doing intermediate visual reasoning steps? [00:16:16] S0: So in the Pi0.7 model, the visual... The the image generation is actually done by a separate model. And this is very similar to how you would... You know, if you load up, like, Gemini or ChatGPT and you ask it to generate a picture, that's going to be calling into a separate model, but the the input to that image generation model is produced by the LM. So it's kind of a similar principle. I think, ultimately, we'll see integration of all these things where there'll be one model that does everything. But, you know, with where the tech is now, the image generation part is usually separate. [00:16:45] S1: What does it mean to be a researcher, like, in kind of the robotics field? What is the day to day of, like, a kind of, like, in a researcher at Physical Intelligence? Like, you know, I think it's interesting. Right? Like, a lot of folks who either have computer science backgrounds or kind of worked as software engineers, you know, a particular mental model of, like, you know, their day to day job. But I'm curious always, like, what does it mean specifically to be a researcher? Like, what is... Like, what what are you doing on a day to day basis?
00:17:15 · Sergey Levine
[00:17:15] S0: So robotics is very, I guess, heterogeneous. Right? So to get a robot to work, you need to do lots of things that will range from, like, tightening the screws that attach the gripper to the arm to figuring out how to train a model on, you know, thousands of GPUs. There isn't really one answer because every day is different, which is one of the things that makes us so much fun. But, you know, honestly, if there was, like, one one trait that I think is really important for a robotics researcher, it's attention to detail. And the reason you need such attention to detail for robotics, even more so than other disciplines, is because the stack is really deep. And if something isn't working, it could be not working at the level of, like, someone didn't attach the connector that goes into the camera properly Mhmm. All the way up to the highest level of, like, well, you know, our model is, you know, it... It's like 16,000,000,000 parameters, but it needs to be 32,000,000,000 parameters, something like that. So it could be anywhere in the stack, and having a really good understanding of many different levels of abstraction is really important.
00:18:14 · Aditya Agarwal
[00:18:14] S1: You know, one of the kind of the classic question that I get from people outside of Silicon Valley is, why are AI researchers so highly paid? And the answer is pretty simple. There's just not that many of them, Right? Like, relative to kind of like the impact of these foundation models is the basic supply demand problem. So I'm curious how many great robotics researchers do you think there are in the world right now that could have that level of attention to detail? And, you know, could like... Is it like a few 100? If there's just not that many people who have done this. Right?
00:18:47 · Sergey Levine, Aditya Agarwal
[00:18:47] S0: It... It's a tricky question because I think there's a big difference between the... You know, saying, like, how many people have a robotics PhD from a top university university and, like, a publication record and the willingness to get stuff to actually work and have that attention to detail as opposed to how many people Okay. Yeah, have the the mindset necessary to get this right. And there's a big difference. Like, you know, there are certainly people that have strong publication record, PhD from top school, but just don't have quite the mindset to go to work. And on the flip side, there are people that have the right mindset, don't have the credentials, but have the ability to get there, you know, in the right environment. So it's a very hard question to answer, and, you know, probably the main lesson for me is, like, not to prejudge these things and to really pay attention to how somebody approaches the problem, what is their, you know, their actual thought process, and what is their, I guess, like, life philosophy for lack of another word. [00:19:39] S1: So I guess maybe to to kinda put you on the spot here, like, do you think it is possible to get hired at Physical Intelligence as a robotics researcher without having a great publication record, but having a great mindset? It's like, listen. I I am a great software engineer. I care about this. I can learn. I have attention to detail. I've kinda done, you know, kind of at least the basics of how all these models work. Do you think that's, like, do you think that's a possibility?
00:19:39 · Aditya Agarwal, Sergey Levine
[00:19:39] S1: So I guess maybe to to kinda put you on the spot here, like, do you think it is possible to get hired at Physical Intelligence as a robotics researcher without having a great publication record, but having a great mindset? It's like, listen. I I am a great software engineer. I care about this. I can learn. I have attention to detail. I've kinda done, you know, kind of at least the basics of how all these models work. Do you think that's, like, do you think that's a possibility? [00:20:02] S0: Yeah. Absolutely. I think the biggest challenge with that is is the the signaling aspect of it. Right? That in any such situation, if you want a job, the other party needs to have some way to evaluate [00:20:14] S1: Yes.
00:20:14 · Aditya Agarwal, Sergey Levine
[00:20:14] S1: Yes. [00:20:15] S0: Yes. And that's very, very hard. And, you know, I I I wish there there was a way to to do that better, but, you know, certainly, the most important thing is not the the the PhD or the credentials. It's the right mindset, the willingness to learn things, to admit that there's some things that you don't know, and most importantly in robotics, the willingness to have the attention to detail. And the attention to detail at at many levels in the stack, it's like easy to say, but it's hard to do because people, especially smart people with an engineering mindset, often have a natural tendency to specialize. And that is a bit antithetical to having that attention to detail to realize that, like, the reason your model is not running is not because you don't have the right learning rate. It's not because you didn't use the right dataset. It's because you ran on a robot where the fingers have a little gap between them when you command them to close fully because somebody didn't attach the connector correctly. So you have to go there, plug in the connector properly, test it yourself, and then go back to training your models.