Latent Space: The AI Engineer Podcast
00:58:25 · Shawn Wang, Jacob Lauritzen
[00:58:25] S1: Any other elements of, let's call it, legal reasoning, or what's the the the legal frontier that models are not good enough yet that you want from the next generation of models? [00:58:35] S0: Nothing I can think of specifically right now. I think long document reasoning is still difficult. Long, like, million token token contracts is what Exactly. You have a huge contract. Yeah. And, like, you're making edits across it. It's a difficult thing to do for a model today even. And, like, remember that you changed all these things up here, and, like, the definitions changed, and that you... The ordering changed, whatever else. [00:58:55] S1: So the simplest would be one shot, you put... Feed everything in and then one shot output. Yes. You don't do that, you break it up into chunks and then you orchestrate edits. Yeah. Exactly. Exactly. [00:59:05] S0: And this is like one of those things that are funny, like, that are fun to be part of in this industry, right? So, word ad, and it was like, we've done, you know, we've done MapReduce, and we've changed, and there was one big one, and there was like one big one in multiple times, it's like, we keep changing the scaffolding that we build around as models change. [00:59:20] S1: Yes. I I would say I I've recently... I do a lot of news reporting, obviously, because of what I do, and I've recently shifted from MapReduce to throwing the time into context. Because it's Yeah. Exactly.
00:59:20 · Shawn Wang, Jacob Lauritzen, Alessio Fanelli
[00:59:20] S1: Yes. I I would say I I've recently... I do a lot of news reporting, obviously, because of what I do, and I've recently shifted from MapReduce to throwing the time into context. Because it's Yeah. Exactly. [00:59:33] S0: But it's also... It's different. I mean, we had this problem with search where it's like, you're searching for certain things in a huge contract, and you can map reduce it, but then it might, if you're summarizing any sort of thing, then it might lose some important nuance in between the chunks, essentially, if you just summarize the individual parts up. So, this has been a problem where it's like, we can do two types of search, we can also do the other one where it's like, you need to have reasoning in the different steps there in between. So, I'm trying to think of a really good example here. Yeah, there's some documents where they talk about something at length over, like, let's say two pages or whatever, and then each individual chunk is not necessarily relevant to the search term or what they're trying to figure out, but together it's extremely relevant. And so, yeah, these are some of the, you know, problems where it's [01:00:15] S2: And it must be hard, like you said earlier, to actually make the models better at that, because you can't even look at the contracts, right? Exactly. Do you guys ever ask them for like, hey, can you make example contracts that are similar to Yeah. The real
01:00:15 · Alessio Fanelli, Jacob Lauritzen, Shawn Wang
[01:00:15] S2: And it must be hard, like you said earlier, to actually make the models better at that, because you can't even look at the contracts, right? Exactly. Do you guys ever ask them for like, hey, can you make example contracts that are similar to Yeah. The real [01:00:30] S0: Yeah. So we feedback, and can receive feedback from them, and if they're willing engage, yeah, exactly. They're just like, hey, this is like super important to us and it's not working well, and then we're like, okay, great. Can you like anonymize the documents and like the problem, whatever, and we'll... If we can recreate it, then we can fix it. [01:00:44] S1: Yeah. I mean, so one version of this is you buy a law firm Yep. Which some people are considering. [01:00:50] S0: Yes. But there's some stuff there on whether their data is actually their data or if it's the client's data. So, if their clients are all gone, then maybe it's fine. I actually don't know. Interesting. Yeah. But it's a... Yes. It's it's a way to get a lot of data. [01:01:05] S1: Yeah. People... Like, Google's buying Spiritualized. Exactly. Right? [01:01:09] S0: Yeah. They bought Yeah. They bought the... Yeah. Yeah. Yeah. [01:01:13] S1: Like, you know, if you have a failed company, you still have your Slack. You can sell the Slack. [01:01:17] S0: Think there's like I [01:01:18] S1: a YC company that buys your failed company. [01:01:20] S0: Yes, there is. [01:01:21] S1: Yeah. Because it's just like, oh, I just want real Exactly. People doing And then, yeah, just was Exactly. Pull stuff out.
01:01:21 · Shawn Wang, Jacob Lauritzen
[01:01:21] S1: Yeah. Because it's just like, oh, I just want real Exactly. People doing And then, yeah, just was Exactly. Pull stuff out. [01:01:29] S0: Exactly. The problem with buying law firm data is, well, you get all the inputs, the real inputs, which is great. The outputs are a little bit more difficult, because you just get what their end result was, and it's difficult to figure out if, like, the way... You have to basically, like, reverse engineer how they got there. So in our product, when we get real traces from really users, it's extremely high value, but if we just got this output, and we don't know if it's good or bad, or how it was created, it's a little bit difficult. [01:01:53] S1: We're gonna wrap up soon. The last thing I'll point people to is also, there's a lot of alpha left on embeddings. You're a rag heavy workload, you're just looking at MTEB and you're taking, like, top model, you're leaving a lot on the table, you should look at Voyage or some of the custom embeddings. You've gone into this. Yeah, yeah, So basically, don't take default embeddings, even OpenAI embeddings, Gemini embeddings, doesn't matter, customize your embeddings. Yeah. Right? Yes. You will you will support that. [01:02:18] S0: Yes. I think that's right. I think that's right. The only thing is you need to make sure that you're multi language. So, us, we we serve so many languages, so that's [01:02:25] S1: I gonna be mean, you're one of of this. [01:02:27] S0: Yeah. Yeah. Exactly. Mr. Au, helpful there? No. Until...
01:02:27 · Jacob Lauritzen, Alessio Fanelli, Shawn Wang
[01:02:27] S0: Yeah. Yeah. Exactly. Mr. Au, helpful there? No. Until... [01:02:34] S2: Well, until now, they just raised a ton of money. Right? So maybe [01:02:37] S0: Yeah. Let's see. You guys They also raised a ton of money. You're be your champions. [01:02:41] S1: Like I [01:02:41] S0: think we need way more model companies out of the... Out of Europe. Yes. [01:02:45] S2: I just wanted to ask you, like, what predictions do you have maybe over the next year for the wine industry? [01:02:51] S1: We asked a little bit of that earlier. I'll modify it to your second question, which is, in two years, will lawyers do their work inside Legora, or will they use other tools like Claude or ChatGPT and call Legora, like, you know, the API. Yeah. Yeah. Yeah. [01:03:05] S0: Yeah. I think both. It really depends. I think if you're an an big enterprise, you might have Claude co work, and then you'll... It's... I think it's similar to, like, how I use Linear through Claude, but I still go to Linear. I think enterprise will do that. I think law firms will just use Legora [01:03:18] S1: Yeah. [01:03:18] S0: Because that's simpler and better. [01:03:20] S2: Yeah. It kinda reminds me of, like, open evidence for, like, medicine. Like, you're you're basically that for law. [01:03:25] S1: Yep. Yeah. Exactly. Encoding is going to a point where people... The labs will cut you off for competing with them, and this has happened twice. Yep. Do you do you think that will happen to law? [01:03:34] S0: No. I don't think so. I think
01:03:34 · Jacob Lauritzen, Shawn Wang, Alessio Fanelli
[01:03:34] S0: No. I don't think so. I think [01:03:35] S1: Because, like, you know, they are releasing legal services. Yeah. [01:03:38] S2: Yeah. Yeah. [01:03:38] S0: They are. I don't... I mean, we obviously talk to them a lot, but it's also... It's like, hey, are you gonna [01:03:44] S1: Your sales reps will say no. Exactly. They don't make the call. [01:03:47] S0: Exactly. That's not their... Right? And they don't even know, like, often what's actually going on. I don't think I don't think that's the case. They have so much stuff to capture in knowledge work or in in sort of enterprises knowledge work generally, that, like, legal has to be part of that, just like data analytics has to, and sales, and marketing, and finance. So, like, they have to do things that are good there, but they're never gonna go as deep as we are, and I think if if Claude takes small clients, small, you know, corporate clients and they do some of their legal via Claude, like, that's that's great. That's fine. But if you're doing a, you know, $400,000,000 M and A transaction, you're gonna want Legora. [01:04:22] S2: I had kind of an unrelated question, but I'm curious, are you guys going to law schools and kinda like getting them to like teach Legora, like in classes? Like, how does that work? [01:04:33] S0: Yeah. It's definitely a program that we are... I think we've started. I actually don't know exactly where we are, but we we do want people in law school to learn to use Legora. [01:04:42] S2: Like, are you seeing that, like, law schools are changing their curriculums a lot or not really?