Latent Space: The AI Engineer Podcast
00:28:24 · Fabian Theis
We've been running a regular NeurIPS competition on prediction problems in single cell data, sometimes for integration data, also for perturbation prediction. That was very well attended, and the cherry on cake was most definitely that ARC competition, which called the thing not just perturbation prediction challenge, but Virtual Cell Challenge. That sound a bit cooler, right? It picked up very well, And but with metrics that are robust, were maybe... But maybe not not that easy. So for... Actually actually our solution in in this this this consortium, where a lot of labs contributed called open problems, was actually that people need to submit their own metrics in addition, so in order to really quantify maybe subtle things that you might not have thought before. The ARC one sort of fixed a bunch of metrics that weren't even so so single cell specific, and they could be just just exploited. So actually, I think the winning model that was then in the end actually sort of crowned as the actual winning model, but essentially it's a flow matching based thing. So based on on this idea that, you know, people pull forward with this flow, with this self flow type of idea, I think shows you that some nonlinear things are in this case needed, so that the actual real winners, or if you sort of remove the exploitation, was actually something that is this is taking some some nonlinearities. And I think nowadays, we do see that just linear or additive type of models are not cutting the the the cake for more complex things, for I think even nowadays, if you go in a simple with few cell lines and you have sort of gene knockouts then, particularly when the effects are a bit subtle, all these sort of more complex models are sometimes a bit overfitting, and that's why linear baseline is always very good. So I always tell my students, guys, I know we we... Guys, I mean guys and girls, we always need to do... I know you wanna do the most fancy deep learning thing, but please, please, you know, have the baseline. And like say for drugs, know, baseline as simple as just predicting getting better at the mean, you know, like that's like the hardest place. Like not even linear model, like better than the mean. Sometimes not as hard if there are so much noise in the data, and this stuff is super noisy.
00:30:15 · RJ Hanaki, Fabian Theis, Brandon Anderson
[00:30:15] S1: I just wanted to add one point that Bo Wang and some of his collaborators actually just had a nature paper on on how the metric was actually the problem in some of the... Those linear baselines beating some of the models as well. So it's clearly... It's not only metric hacking, but also just that the asking the wrong question leads to the wrong answer. [00:30:35] S0: Or choice of the metric. Yeah. And I I think that they could... You can discuss it. At some point, we're gonna have agents sort of trying out all kind of combinations of whatever algorithmic thing you can come up with. In fact, we actually do have some collaborations with some of of those sort of AutoML couple labs to try this out. So maybe what is left sounds a bit too harsh. Right? But like, boy, maybe our biggest ideas go into it is actually this method formulation, RJ, right? [00:30:58] S2: So one thing I've been really interested in is the different models of funding and research academia. I think Germany has a different structure fundamentally from The US. How is your institution set up? And like, how does this give, like, distinct opportunities versus like a US based research model?
00:31:15 · Fabian Theis
[00:31:15] S0: I I very much agree that these funding models very much matter for the flexibility, but also the set of teams that you can set up and like the long term vision that you can have. So Helmholtz, this is not just Germany, but I would say maybe Europe's largest, what's called research organization in terms of funding. It has a different set of areas that it focuses on, health being one of the big ones, but then it has sort of material information sciences, physics, and so on. And essentially getting fed federal money to put into a deep research question. I've been building up what's called the computational health center at Helmholtz Munich, which I think has now something like maybe 45 group leaders and professors, and maybe 500 to 600 people now. So we are, I think, the largest academic place for health AI type of research in Europe, where we have people really working on all scales. Like, you know, I'm sort of focusing on the cellular scale, but, you know, there's people doing DNA foundation models. There's people sort of doing interesting things on protein design. There's people doing like, you know, patient cohort type of representation, genetics, and so on. And now you sort of can start putting them together, and one of our big visions is sort to to intersect and to go virtual human and so on. But having said that, the key thing is in contrast to universities, that even though we all have affiliation to universities, mostly to the Technic University or the University of Munich, the two top ranked universities interestingly in the European Union. So we have this sort of place where we have these affiliations at universities. Right? But we don't have undergraduate teaching. We essentially just have PhD students and postdocs, but we interact very much with the universities and in particular also the clinics because you wanna have sort of this translational thing in there. And increasing... That's maybe an interesting link. So this is maybe a bit similar to NIH type of models like in US, but sort of distributed all over the country, not just sort of in one place.
00:33:00 · Fabian Theis
And maybe the one interesting link is that increasingly the government and this sort of realization also in in Germany is is sort of dawning that there needs to be much stronger transfer component, not just by going into translation to universities, but also in into sort of value creation outside of the academic realm, which is obviously going so more more into startups and so on, which I think the Boston environment has always been one of the key examples, but now, of course, more so now also the the the Bay Area when, you know, recently I was visiting, I believe maybe Genentech or colleagues, and we went to a restaurant, and then one of the waiters was asking us, hey, are you guys talking about AI and bio, you know, I have been also interested, so like everyone's sort of thinking about this startup ideas and so on, and I I kind of felt that's a it's a good environment, and this is something I think our PhDs are now more and more doing, and and we were sort of learning there, and I think also in terms of sort of value creation, Munich in particular is actually at least in Continental Europe, maybe the prime place now for VC capital, but also sort of deep tech, particularly the bio and and defense area sort of pretty exploding. So this stuff starts to work.
00:34:04 · Brandon Anderson
[00:34:04] S2: One thing... One... The thing you were talking about startup, I think, really fascinating. I've heard that Germany has had some issues with their startup ecosystem. It's it's had... It's difficult to create a startup there compared to even other places in Europe. How... Like, what is the competitive landscape for startups right now in Germany and specific... And how does Germany become more competitive broadly? I I have a vested interest in this because I'm I'm probably gonna get my German citizenship soon.
00:34:31 · Fabian Theis
[00:34:31] S0: Welcome. Let me know if you come come by Munich. So I think traditionally, Europe... But I would say definitely Germany is is quite bureaucratic, and processes are far from as digital as you would expect in different places. Not that I'm saying that US is maybe the best example for fully digitized processes, but stuff is definitely faster. The thing is people have realized that now, and I think the the particular startup creation and implementation is, I think, more than competitive in Germany versus Europe. It's actually sort of the the the prime number, and in particular, like some hotspots, Berlin, but in particular, Munich sort of the... I I think is is the capital for for an... Highest number of startups. And I think even nowadays, if you look into European Union, there's equally as many startups being created as in US. The thing is where Europe and particularly also Germany drops off is at the scaling part. So, you know, early VC and also sort of non dilutive type of money, this is there. But then when you go into scaling, these big VCs that sort of take these bigger risks are, I I think, not not closely as present. So I've I've I've saw this again and again that, you know, if you wanna sort of really go into the the sort of bigger tickets sort of series A and so on. The series B, you... People start going more into US also for that. But maybe I'm sort of simplifying here, but this is at least my current learning, Brenner.