01:10:30 · Keller Cliffton, John Coogan, Jordi Hays
[01:10:30] S3: And it's really probably only two options. Yeah. And so, you know, it's really... I don't know. It it inspires me. Like, this is an area where America is winning, and and there's a lot of hard work to do, and to see the technology now scaling very quickly in cities across The US, and you know, benefiting families, letting families spend more time with their loved ones, less times having anxiety about errands. Totally. It's going have a big impact. [01:10:52] S1: I love it. Thank you for what you're declaring order inbound. Backing up the truck. Great talking to [01:10:58] S3: have speaker we have a speaker on the next gen droid. Should we like play the Call of Duty? [01:11:03] S2: Definitely need... Sound files... After you [01:11:08] S1: get your second delivery, it hits... Scrappy. Another one. [01:11:11] S2: And then I think you might... It might be a little too scary for the general population, but you can... We... If we can set that in our preferences. [01:11:17] S1: Successful delivery? Yeah. [01:11:20] S2: Okay. We got another. Keller, give our team the best. Incredible progress. Can't wait for the next update and we'll we'll see... We'll we'll stop by [01:11:28] S1: next [01:11:28] S2: time Goodbye. We're in the [01:11:30] S3: Awesome you guys. Thanks. Ad[01:11:31] S2: Good man.
01:11:31 · Jordi Hays, John Coogan, Rahul Sonwalkar, Praveer Melwani
Ad[01:11:31] S2: Good man. Ad[01:11:31] S1: King in the castle. King in the castle. Let me tell you about public. Invest across all asset classes. Automate anything with AI agents, run trading strategies, risk management, money movements, and monitor markets with AI agents for investing on public. Speaking of finance. Speaking of finance. Hi, finance. We got Rahul Sonwalkar from Julius AI. [01:11:52] S2: There he is. [01:11:52] S1: Fucking absolutely diced and shredded as always. How are doing, Rahul? [01:11:56] S6: Doing fantastic. How are you guys? [01:11:58] S1: We're doing great. [01:11:59] S2: It's great to see you. [01:12:00] S1: Always good to have you on the show. Take us through the latest in your world, in Julius' world in San Francisco. Take us wherever you wanna go. [01:12:08] S6: Alright. So we just released Deep Finance. Mhmm. It's the first ever long horizon high finance benchmark. [01:12:16] S1: Okay. [01:12:17] S6: So the goal is to see how good models do on real world big deals that you would see in investment banking [01:12:25] S1: That's good. And [01:12:25] S6: private equity. [01:12:27] S1: We love this. [01:12:28] S6: It's giving... Money users. Exactly. [01:12:30] S1: Yeah. [01:12:30] S6: So we add... We we work with experts who have experience from firms like Evercore Sure. JPMorgan UBS, and Harvard Business School [01:12:39] S4: Yeah. [01:12:39] S6: To help build this benchmark so we can understand how good the frontier models are. [01:12:44] S1: Yep. [01:12:44] S6: On an actual real deal.
01:12:44 · Rahul Sonwalkar, Jordi Hays, John Coogan
[01:12:44] S6: On an actual real deal. [01:12:46] S2: Yeah. So... So how does it like like... It's very cool, but like I'm very curious how this... Like the the complexity in a deal ends up coming, you know, you... Let's say post term sheet, I don't know where you kind of try to start in in... [01:13:03] S1: Yeah. Where does this start? Where does it end? [01:13:05] S2: The complexity comes from like the, you know, emotional dynamics between all the different individuals involved and combining all these different preferences. And so how... Did you have to like simulate the interactions that come after you go from like a term sheet into the actual, you know, let's say like purchase agreement in the case of, you know, an m and a deal?
01:13:05 · Jordi Hays, Rahul Sonwalkar, John Coogan
[01:13:05] S2: The complexity comes from like the, you know, emotional dynamics between all the different individuals involved and combining all these different preferences. And so how... Did you have to like simulate the interactions that come after you go from like a term sheet into the actual, you know, let's say like purchase agreement in the case of, you know, an m and a deal? [01:13:27] S6: Exactly. So what we start with is real bare bones data room with a lot of messy data. Mhmm. You know, what what does an actual deal look like? An actual deal looks like, hey. We are looking at this company. Here's a bunch of files and a bunch of data. Go figure out if it's worth pursuing this deal. Mhmm. What does that look like? Can we take this to the investment committee? Mhmm. It's a super high level open ended instruction. Mhmm. And we wanna evaluate these models on is, well, can they fully understand the business Mhmm. In a way that a private equity or investment banking professional would? Do they just look at revenue and just assume that, okay, that's the revenue? Or do they actually go trace the revenue down to the to the numbers, into the sources, and see if it all adds up? Mhmm. In real world, you see a lot of discrepancy. [01:14:17] S1: Yeah. [01:14:17] S6: You see, like, the management report one number, and the audit report another number, and then we actually do your own math, you find out that it's something else. [01:14:24] S1: Yeah.
01:14:24 · John Coogan, Rahul Sonwalkar, D.A. Wallach
[01:14:24] S1: Yeah. [01:14:25] S6: And the models deal with that complexity. Mhmm. So these workflows are actually designed to to reflect that robot complexity. So there's, you know, a lot of emails in the tasks. There's raw data. There's some financials. And then the models are tasked with coming up with a... You know, financial model, like a DCI, an LBO model Sure. And come up with a memo to go pursue a deal or not. [01:14:51] S1: How do you think about the the business impact for Julius on this? I mean, seems like you're getting close to maybe you already have built like an RL environment around this. You could sell that to a big lab. You can use it to partner with other model companies to fine tune things. You can integrate directly into your product, build a harness around it. Like, there's so much that you can do once you have the data, once you understand the problem. How are you planning to to actually deploy it for financial gain? [01:15:19] S6: Absolutely. To be honest with you, don't know. Okay. So we just launched Julius research. And the goal of Julius research is to go pursue interesting problems that are relevant to Julius and we think are gonna help impact the model progress on on on knowledge work. [01:15:34] S0: Mhmm. [01:15:34] S6: Right? And so one of the things we did with with this launch is we specifically did not measure the models on our own harness. We use the native CLI. So for open AI models, we use codex. [01:15:45] S1: Sure.
01:15:45 · John Coogan, Rahul Sonwalkar, Jordi Hays
[01:15:45] S1: Sure. [01:15:45] S6: For Claude models, we use Claude Code. Yep. And just to be independent and really understand where the models are. Mhmm. And so that's the first step, is push with the curiosity and see if we can build the first ultra long horizon benchmark for finance workflows, and we did. And now what we're doing is understanding where do those different models stack up, what are the different idiosyncrasies in how these models pursue these tasks. Yeah. So one of the things we noticed is models that are really good at coding, you know, they tend to search a code base. Right? They didn't search a code base to understand it. That doesn't always work in finance. Yeah. If you just search for revenue in the documents Yep. And you see a number, you can't just assume that's the revenue. You have to see if it all adds up. [01:16:31] S1: Yeah. [01:16:31] S6: And so... [01:16:33] S1: Well, many many modern deal makers aren't even looking at the revenue. They just ask, like, who else is in over a text message, and then that's their criteria. So you you should fine tune a model on that. They just ask ask ask who's who's leading and then and that's the entire that's the entire criteria. [01:16:50] S2: Oh, you want me in the round? Which round?