Latent Space: The AI Engineer Podcast
00:12:40 · RJ Hanaki
[00:12:40] S1: That's a great connection to the the organoids, idea. So can you talk a little bit about... Okay. So an organoid is is what? And and how does that connect to this concept of perturbation space? And what do you do to it in addition to adjust this context?
00:12:58 · Fabian Theis
[00:12:58] S0: Dude, when I when I first heard about... Learned about this concept, I should say, you know, I'm I'm an auto trained biologist. So I typically learn about things when I read a bit or when I talk to colleagues and and friends. And when I first saw that people are able to take essentially cells out of a body, out of a cancer cell or whatever, right, and let them then grow into small multicellular systems that actually sort of start, connect, and do things. I was super super impressed. So, you know, people can now, for example, take some cells, put them in a dish, and then grow little mini brains, essentially set of neurons that essentially really start connecting, talking to each other, and people have shown that they can get those sort of mini brains in a dish to compute. Similarly, people can take heart cells, grow them into little heart organoids, and we have a bunch of local corporations with fantastic partners that actually start beating. Like, you have beating little heart cell clumps in a dish, right, and then of course, once you have these, and you're same thing as I spoke about brain, there's even simple systems like the colon where this works very well. There's a whole bunch of cancer systems. But you know, also for... Let's say if you wanna study diseases with hard to acquire material, let's say you wanna look into the genetics of some type of of neurodegenerative diseases or so. Actively, it's very hard to get access to tissue. Right? So you take stem cells from the skin... You take cells from the skin, differentiate them, make... Turn them into these mid little brains, and then you can potentially study some some things there. So I think this concept of replicating little bits of organs, that's why they call organoids, in a dish is very fascinating. There's all kinds of limitations, obviously, of an immune cell... Immune system is missing, there's like no, maybe feeding them or something like that. So there's all kinds of problems, right? But it's much, much more than just a cell line, which is like one cell type.
00:14:34 · RJ Hanaki
[00:14:34] S1: Moving to the modeling side, how does... How do you model that?
00:14:37 · Fabian Theis
[00:14:37] S0: So that's, I think, bit the research frontier. So I don't think people have agreed on what the optimal way to waddle is, particularly because there's not as much data as one wants. Jay, right, you're involved, if I understand correctly, in a whole bunch of spatial omics slash also sort of H and E sort of pathology type of effort. So like pathology, there's like so much samples available, right? And now increasingly we sort of do a bit more spatial omics. So I think how we even represent spatially connected data sets, and then you sort of break it down on an organoid level and so on, people are not as clear as it is with disassociated cells. So again, taking this a bit simpler, I think if we look at the space of disassociated cells and we look at these groups, we have this beautiful metaphor of what the humans call a cell type, which you know nature doesn't care about, This is just us putting categories on top of stuff, but it helps us understand this complexity. I don't think we have a similar metaphor for local niches of cells, maybe a little bit, but also in H and E, right. Whenever I talk and see what pathologists do, it's often very qualitative and it's maybe not at that level yet. So I think we need to develop a bit of a categorization language for us. Maybe computationally, maybe we don't need this so much, but computationally we also need to sort of think a bit how we deal with this. At the moment, we essentially deal with this as distributions of of of cells. So this is sort of the solution at the moment, but I think it can be more much more subtle, like graphs or something like that.
00:16:04 · RJ Hanaki
[00:16:04] S1: So on the one hand, you have the bitter lesson and, you know, sort of you, you remove inductive bias, and you end up being able to have a larger data set that's applicable to your problem. And then that helps you to get more accurate models because your, the your distribution is wider and whatever. So that's like the one and then but then there's this, like, countervailing tendency, which is like, maybe you could call it the anti bitter lesson, or like the other the flip side of the bitter lesson where when you're in data poor regimes, for one thing, then you have to induce introduce a lot of inductive bias to get enough statistical power to make good inferences. But there's also a second side of that, which I think John Platt in our discussion with him not too long ago, brought up was that you have these these, like, predictive and then descriptive models. And what you really want as a scientist is to get to a simplification, a descriptive model where you can say, oh, well, I've tested all the apples and the gravity works in such a way on apples. Then but I'm modeling this as a universal concept simply so that we know that it works on plants as well. Right. And and so that I want to understand from you, what are the simplifications or, you know, sort of maybe simplifications, maybe insights, maybe actual mathematical or other wise scientific models that are becoming apparent that are gonna help to reduce all the complexity so that we're able to actually make meaningful... Gain meaningful understanding from these really large datasets.
00:17:39 · Fabian Theis
[00:17:39] S0: Yeah. That's that's a... That... That's a very important... Very... I think kinda deep point, which I think sort of breaks the field into maybe sort of a bit two different groups of people with answers. Right? So how much do we need to put inductive biases maybe priors into a model to maybe make it better explainable, right? But potentially also more robust in going out of distribution? Or how much are we gonna just rely on bigger data to sort of have the model be more expressive and so on? And I think... And I need to be very clear before we may go into the philosophy. I think literature has really clearly shown over, I think now decades, at least for our very many data sets, that if we build an a priori inductive biases too strongly, that we might lose predictive power, right? So it comes at a cost. So if you're now, let's say, a physician, and you just want to... Or maybe even if you're a patient and you you maybe you might have cancer or not, maybe you want to have the best model as a predictor. Maybe she getting treated, you want to have a model that is maybe a bit less good, but you understand it a bit more. But for the prediction, at least I think you you want to have the best. And this is I think I think not entirely clear. And I think in our field with this perturbation space scale, the sales guy I think is big, but the perturbation space scale, like the stuff that we do to it not being as big, I don't think we're yet in the super data rich regime. So I think in our case, these hybrid models might be still the thing to go for. But more and more, of course, and this is I think this, I don't know if you can call it sad, but the honest learning from from big scale machine learning is that at some point data beats human made additional concepts in these models. And I have at least one simple example I often put is AlphaFold, where, you know, the first model sort of made this extraordinary effort to to make, you know, the molecules rotationally variant, they sort of build it into the learning process, and only in later versions, you know, sort of rotation symmetry was automatically found by the model just with more data, it was a bit more flexible because whatever physical constraint you put in, it's not always as rigid, and then you sort of you gain something if you let the model learn from this.