01:12:44 · Rahul Sonwalkar, Jordi Hays, John Coogan
[01:12:44] S6: On an actual real deal. [01:12:46] S2: Yeah. So... So how does it like like... It's very cool, but like I'm very curious how this... Like the the complexity in a deal ends up coming, you know, you... Let's say post term sheet, I don't know where you kind of try to start in in... [01:13:03] S1: Yeah. Where does this start? Where does it end? [01:13:05] S2: The complexity comes from like the, you know, emotional dynamics between all the different individuals involved and combining all these different preferences. And so how... Did you have to like simulate the interactions that come after you go from like a term sheet into the actual, you know, let's say like purchase agreement in the case of, you know, an m and a deal?
01:13:05 · Jordi Hays, Rahul Sonwalkar, John Coogan
[01:13:05] S2: The complexity comes from like the, you know, emotional dynamics between all the different individuals involved and combining all these different preferences. And so how... Did you have to like simulate the interactions that come after you go from like a term sheet into the actual, you know, let's say like purchase agreement in the case of, you know, an m and a deal? [01:13:27] S6: Exactly. So what we start with is real bare bones data room with a lot of messy data. Mhmm. You know, what what does an actual deal look like? An actual deal looks like, hey. We are looking at this company. Here's a bunch of files and a bunch of data. Go figure out if it's worth pursuing this deal. Mhmm. What does that look like? Can we take this to the investment committee? Mhmm. It's a super high level open ended instruction. Mhmm. And we wanna evaluate these models on is, well, can they fully understand the business Mhmm. In a way that a private equity or investment banking professional would? Do they just look at revenue and just assume that, okay, that's the revenue? Or do they actually go trace the revenue down to the to the numbers, into the sources, and see if it all adds up? Mhmm. In real world, you see a lot of discrepancy. [01:14:17] S1: Yeah. [01:14:17] S6: You see, like, the management report one number, and the audit report another number, and then we actually do your own math, you find out that it's something else. [01:14:24] S1: Yeah.
01:14:24 · John Coogan, Rahul Sonwalkar, D.A. Wallach
[01:14:24] S1: Yeah. [01:14:25] S6: And the models deal with that complexity. Mhmm. So these workflows are actually designed to to reflect that robot complexity. So there's, you know, a lot of emails in the tasks. There's raw data. There's some financials. And then the models are tasked with coming up with a... You know, financial model, like a DCI, an LBO model Sure. And come up with a memo to go pursue a deal or not. [01:14:51] S1: How do you think about the the business impact for Julius on this? I mean, seems like you're getting close to maybe you already have built like an RL environment around this. You could sell that to a big lab. You can use it to partner with other model companies to fine tune things. You can integrate directly into your product, build a harness around it. Like, there's so much that you can do once you have the data, once you understand the problem. How are you planning to to actually deploy it for financial gain? [01:15:19] S6: Absolutely. To be honest with you, don't know. Okay. So we just launched Julius research. And the goal of Julius research is to go pursue interesting problems that are relevant to Julius and we think are gonna help impact the model progress on on on knowledge work. [01:15:34] S0: Mhmm. [01:15:34] S6: Right? And so one of the things we did with with this launch is we specifically did not measure the models on our own harness. We use the native CLI. So for open AI models, we use codex. [01:15:45] S1: Sure.
01:15:45 · John Coogan, Rahul Sonwalkar, Jordi Hays
[01:15:45] S1: Sure. [01:15:45] S6: For Claude models, we use Claude Code. Yep. And just to be independent and really understand where the models are. Mhmm. And so that's the first step, is push with the curiosity and see if we can build the first ultra long horizon benchmark for finance workflows, and we did. And now what we're doing is understanding where do those different models stack up, what are the different idiosyncrasies in how these models pursue these tasks. Yeah. So one of the things we noticed is models that are really good at coding, you know, they tend to search a code base. Right? They didn't search a code base to understand it. That doesn't always work in finance. Yeah. If you just search for revenue in the documents Yep. And you see a number, you can't just assume that's the revenue. You have to see if it all adds up. [01:16:31] S1: Yeah. [01:16:31] S6: And so... [01:16:33] S1: Well, many many modern deal makers aren't even looking at the revenue. They just ask, like, who else is in over a text message, and then that's their criteria. So you you should fine tune a model on that. They just ask ask ask who's who's leading and then and that's the entire that's the entire criteria. [01:16:50] S2: Oh, you want me in the round? Which round?
01:16:50 · Jordi Hays, John Coogan, Rahul Sonwalkar
[01:16:50] S2: Oh, you want me in the round? Which round? [01:16:52] S1: Which one? Am I in the trunch? We'll figure it out. Yes. Can you get me up to speed on the latest and greatest in Thinking Machines world? Sort of a quiet company. There's a leak that it sounds like they're they're closing around 5,000,000,000 at 40, incredibly impressive. Some of the open source models have been really good. I heard that they're the biggest partner to to Reality's fund, believe. Is that right? But it seems like there's some traction there. You have some data. What's the latest in TML world? [01:17:24] S6: Yeah. So we actually... What's interesting is that they released InkLink too small yesterday [01:17:29] S1: Yeah. [01:17:29] S6: Which is the same day we released our evals. Yep. And so our evals weren't public when they trained this model. [01:17:36] S1: Interesting. [01:17:37] S6: And when we ran their model on our eval, we found that it's actually the best American open source model Yeah. And second best open source model overall [01:17:47] S1: Yeah. [01:17:48] S6: After Kimi. And so that's really, really impressive. Yeah. Especially if you see the leap from Inclink one to Inclink two Yeah. In a matter of months, that's really impressive. I think things are really really starting to work there. And especially if they're working with Reality's fund [01:18:08] S1: Yeah.
01:18:08 · John Coogan, Rahul Sonwalkar
[01:18:08] S1: Yeah. [01:18:09] S6: I can see why understanding how good their models are at finance Sure. Is important. And also, it's open source. So that doesn't mean that this is just how good Eagling Too Small is gonna be. You can actually both train it to be even better. [01:18:23] S1: Yeah. And a lot of financial institutions might want to have data sovereignty. They might already already... They might already have their own infrastructure in many cases. And so it's interesting that you're seeing like the bleed over of whatever they're doing with their very secretive financial partners that they don't talk about. It's sort of leaking in their open source models which is great because then that's something that everyone can benefit from and it also serves as top of funnel. Exactly. To It's see. Well Exactly. I mean Yeah. Please. No. No. [01:18:54] S6: I was gonna say, you know, especially with American open source models, I think that ecosystem really needs to catch up. Yeah. And it's so impressive to see that it's even better than Nemotron from NVIDIA. Yeah. Because that gives institutions, even government agencies, and private companies more conviction that they can actually both train these models and and not... Yeah.