Financial Modeler's Corner
00:19:47 · Paul Barnhurst, Peter Fuller
[00:19:47] S2: I have a couple questions and I'll let Anand Giles go again. The first, when you mentioned, you know, the models, I'd love to get your thought on... Are they gonna all kind of become a commodity at some point? The models will just all be pretty similar. Where do you think we're heading? Because it feels like outside of customized special, the big huge ones, how much can you differentiate yourself in a model when you're all training off similar data? So I just love a little bit of your thought there. You're already smiling, so I know you have an opinion. [00:20:17] S1: I'm I'm not gonna claim to know the answer. I don't think anyone knows the answer to that question. [00:20:20] S2: Sure. I just love your thoughts. [00:20:22] S1: I mean Yeah. [00:20:23] S2: Yeah. If you're right, we'll come back and congratulate you. And if you're wrong, you'll be like most of
00:20:26 · Peter Fuller
[00:20:26] S1: us. Yeah. Yeah. Well, look. I'll take a couple of things. The commodity thing, I mean, it's kind of baked into their mission statements. Right? Like, OpenAI always talk about intelligence too cheap to meter. And I I read that and I think, is that good for your business model? Like... But they're always talking about that. Right? Well, that means it's a commodity. Right? So I think I think there is there is gonna be a lot of that and people will make a lot of money selling the compliments to that commodity. Right? Our customers included. It's already happening, by the way. Right? As as the power of the models grows, this intelligence grows, our customers, like our consulting customers, I think BCG grew AI transformation related revenue at, like, 25% last year, something crazy like that. And they're just making so much money implementing AI in, you know, helping customers implement AI. So you're already seeing that effect whether... With the compliments are doing really, really well. The other thing is, I mean, I I I really I really don't know. There's been, like, architecturally, there's just been so little diversity in the frontier models. I mean, everyone's kind of pursuing the same thing, right, and scaling it. That's that's been the winning play for the... That's how many years. I don't know. It's perfectly possible that startup or someone like Ilya releases a model that is architecturally different and has very different capabilities, maybe different spikes and troughs versus the existing models. And that would be amazing for companies like us. Right? Diversity at the model layer increases the value of orchestration between those models. Right?
00:22:23 · Giles Male, Peter Fuller, Ian Schnoor, Paul Barnhurst
[00:22:23] S4: And Because you... Because your users can choose between them. [00:22:27] S1: So easy. I mean, it would be so... Let's say, Ilya releases this, like, fantastic model, which is just very different from from Anthropic models. And then I... You know, customers would be able to use Tracelight, and we would use both the models for different tasks. Right? And so the value of orchestration would grow. But people have been thinking that for a long time and it hasn't happened and it might not happen. Right? It might just be scaling all the way. I don't know. [00:22:55] S4: And just to delve on that, because the way you just used the word orchestration, I'd never thought of. Because to me, I was thinking sort of within, I don't know, like a cloud environment, you have a sub agent team of agents. But you're actually talking about the orchestration of potentially the use of different LLMs [00:23:15] S3: We already know. [00:23:15] S4: Within this... Yeah. Okay. Oh, wow. Okay. [00:23:17] S2: Well, think about... If I can say one thing, I think, Giles, when you think about it, you're selecting which model to use within an LLM. They're just selecting which model to use of all LLMs. [00:23:30] S4: But but but but also by the sounds of it, but this is what the... I wanna just make sure I understand. Orchestration for me would be like within a a a a workflow, you're actually delving within different LLMs from different providers within a workflow.
00:23:30 · Giles Male, Peter Fuller
[00:23:30] S4: But but but but also by the sounds of it, but this is what the... I wanna just make sure I understand. Orchestration for me would be like within a a a a workflow, you're actually delving within different LLMs from different providers within a workflow. [00:23:44] S1: Yeah. So for example, if you upload 500 pages of PDFs into Tracelight and say, here's contracts, read contracts, transcribe, and build a model. Right? It would be Gemini reading the... Reading those PDFs, and then it would be today, Claude, doing the modeling for you. Right? So it already happens in our in our product. And, you know, on our kind of pro tier as well and our max tier, you can choose between using the kind of frontier intelligence. Right? Or we have efficient mode as well where we'll use, yeah, like a smaller cheaper model. Right? But people get way more usage out of their out of their subscription. And today, those are from different providers. [00:24:36] S4: Do you have a strong view on... And again, this has come up in a couple of places recently. Do you have a strong view on, you know, choosing the most powerful model out there at the moment versus having a really strong harness and set up, however that is kind of implemented. Do you get a sense of like what is better if that's fair?
00:24:36 · Giles Male, Peter Fuller
[00:24:36] S4: Do you have a strong view on... And again, this has come up in a couple of places recently. Do you have a strong view on, you know, choosing the most powerful model out there at the moment versus having a really strong harness and set up, however that is kind of implemented. Do you get a sense of like what is better if that's fair? [00:24:59] S1: I mean, I don't think we've yet reached the, like, the point where the tasks where you care about are saturated by the behind the frontier models. Right? So, I mean, if you are building a big complicated spreadsheet, you definitely do wanna be using Opus five or 5.6. Right? And, you know, I... We will get to a point, I think, where the saturation will occur, and it it'll probably happen quite fast. Right? Uncomfortably fast. Where, like... Yeah. I mean, Opus five is really, really strong. And so when you've got when you've got Opus six or 6.5, I I don't know if it's worth spending all the money on the on the tokens to get six or 6.5 to write your your formulas. Right? You should be getting it to do something else. So... But we we haven't reached that point yet, in my opinion.
00:24:59 · Peter Fuller, Ian Schnoor
[00:24:59] S1: I mean, I don't think we've yet reached the, like, the point where the tasks where you care about are saturated by the behind the frontier models. Right? So, I mean, if you are building a big complicated spreadsheet, you definitely do wanna be using Opus five or 5.6. Right? And, you know, I... We will get to a point, I think, where the saturation will occur, and it it'll probably happen quite fast. Right? Uncomfortably fast. Where, like... Yeah. I mean, Opus five is really, really strong. And so when you've got when you've got Opus six or 6.5, I I don't know if it's worth spending all the money on the on the tokens to get six or 6.5 to write your your formulas. Right? You should be getting it to do something else. So... But we we haven't reached that point yet, in my opinion. [00:25:59] S3: I mean, listen. I love... I I can't even imagine waking up every day and wondering, is there gonna be an announcement that is just, like, you... Do you even sleep? Like, every hour, something can happen that might, you know, cause you to, you know, flip over or, or... You know, mean, like, it's just... I can't imagine being in a place where... That is changing so fast, but surely, you... I mean, you must be feeling you're doing the 100 meter sprint every day, and wondering what's the newest release. What's that like? Talk about what that's like wondering... Yeah. Obviously, anticipating any days a competitor or a a frontier model might release something that's different. What's the feeling like to be in that environment right now? I