00:11:10 · Brett Caughran, Khe Hy
[00:11:10] S0: Yeah, so the Claude code, like the December, January moment was like a big inflection in the agentic revolution. And even people talk about Claude Code and an IDE, but even the things you can do in Cloud Chat are now agentic. I can go build Excel, you know, build Excel things just directly in Cloud Chat. So Cloud really, I think, took this big step ahead from OpenAI, and you've seen more agentic tools. It's really just the last ninety days, there was like, call it 5% of my workflows, five to 10% of my workflows as an investor was the TAM for AI in '25. Now I feel like it's pushing up to 30, 40, 50, 60%. And the imagination you can apply with these tools now is well beyond what you could do with chatbots. Now there's still some issues that we'll explore this chatbots. Issues, many jagged edges, context window limitations haven't gone away. Multi document retrieval accuracy hasn't completely been solved. But the pace of evolution just in six months is, I think, well ahead of the bull case I envisioned, you know, summer twenty five. [00:12:34] S1: Can I ask you because I feel like the the word agentic gets used a lot? And what is how do you personally define the word? And then I'm happy to share my definition.
00:12:45 · Brett Caughran
[00:12:45] S0: Yeah. No, I'm curious yours. I, I think about the investor toolbox as like workflows, kind of end to end workflows. Hey, I want to go do an earnings preview process or this company has a new CEO, I want to go and explore the background and the potential of this CEO. And within that, there can be 20 steps along the process. I want to go understand or her stock grant, past performance, do a background analysis on if this person has nefarious historical affiliations, etc. In the past, I had to create each of those as individual prompts because as you know, a prompt past five or six pages started to lead to a garbled output with chatbots. So I couldn't build a workflow to do all of these things. And many of the tool providers tried to do that, chaining together prompts and building all of this complex engineering around it. Now I can one shot that an agent, So with a skills it really just makes the tools much more user friendly. It gives me an end to end workflow that still isn't perfect, but is shocking in terms of the quality and caliber. So my three zero six prompts have gone down to 30, really like 18 orchestrated pipelines and then individual skills that I can pull off the shelf to dig in more deeply. So you're seeing this abstraction in the complexity of working with these tools. And there's been innovations around MCP and MCP and data as well, too, that sort of the one example you'd see that if you went into ChatGPT last year and asked a question, often the quantitative metric will be pulled from a blog somewhere, right? So if I went in and I said, what is the ten year history of Nvidia's earnings per share? These were exercises that a chatbot couldn't answer accurately. So it's like, if I can't answer a basic question like that, like get out of my face with this AGI thing. But now I can plug Claude Cowork into an MCP from Dilipa or Faxit or FinShat, and that data is piped in through the MCP process with, in my experimentation, much better accuracy. So it's not a solved problem, but there's been huge innovation. Do you think about agentic workflows and what agents unlock?
00:15:29 · Khe Hy
[00:15:29] S1: Think similar to what you said, so I hear kind of like end to end workflows being one versus kind of constantly steering the model. Okay, now tell me this. Now tell me this. The analogy I use is it's like the difference between Waymo and Google Maps, right? Waymo, you give it the destination and then you just kick back. You don't really care how it gets there as long as it gets you there within a standard deviation of your expectations for time and safety, right? And so with the AgenTic, it's like, hey, I would like to, you know, write this deck. But I don't really care if you do slide seven first or this, but I just have a general intent of what I'm looking for. And then the model kind of figures it out. Right. So that that would be kind of like the end to end. And obviously, you mentioned the connections as well. The other thing that I would add with agents is that they tend to, like, self correct along the way. So you might give it a PDF that's too big. And then without telling it, it will run like a PDF extraction tool, right? Like, I don't even know that the PDF extraction tool exists as a thing. I don't even know that PDF's too big. But the agent is making these decisions and what, you know, in AI language, they'll say tool calling, right? And not just tool calling like Slack or Dulupa, tool calling like these weird kind of utility help error apps to, like, chunk a PDF or to use what we would call Command F, you know, is called grep in, you know, in computer science land. And so the agent not only can it get you answers, but it can find and solve its own problems by accessing new tools, which again gets to that point that you mentioned. It makes the end to end workflow more accurate and longer running. So in many ways, that kind of full agreement with what you said, I would add kind of the tooling element that is kind of abstracted away that, like, I don't have to tell it to use the PDF extractor and then the ability for it to reason and act and self correct in the process.
00:17:42 · Brett Caughran
[00:17:42] S0: Yeah. And I'd add probably one of the it's not the flashiest AI pilled moment, but one of the big AI pill moments I had this year was the usability of agents for validation. And so we work in an industry where accuracy is critical. We have a fiduciary duty to our investors to conduct a rigorous investment process. And if you follow any of the finance benchmarks of LLM accuracy, they tend to sort of shake out in the 60 to 70% range. So I'm always like, all right, well, yes, for qualitative use cases where I'm putting in six transcripts that I can go and validate in the transcript, sure, Right? But for quantitative use cases, that was a very difficult mental hurdle for me to get over. Like I don't want to unleash LLMs on financial models where it's 65% accurate. Exercise of Like finding 35% error is like next word prediction, next token prediction, next word prediction, it's fine. Next number prediction is not fine. We don't want a stochastic exercise to sort of make up a plausible sounding number, right? And in a few of these use cases, like LLMs are very good at making something look plausible, which means that where I've seen quantitative errors, sometimes a quantitative error looks more plausible than the actual number, right? And so this was something where I was like pump the brakes on quantitative use cases in 25. So what's changed is MCP, the data's gotten more accurate. But the ability to build agentic, I was using is like, okay, let's check these numbers, use these four LM to check this LLM and go back and forth. And it was okay, but the ability to create agents to validate work now, it's exactly you pointed out to sort of go in tool call, check all these numbers, right? Even give me a checklist by hand to go in and spot check numbers that could really be problematic for my thesis. That to me is a big unlock with agents that didn't exist in chatbots. Curious, Kay, to understand a little bit more about on your consulting side, are there three or four common themes you see come up from clients in terms of the problem space that the firms you work with have in general that you help them walk through?
00:20:20 · Khe Hy
[00:20:20] S1: I think one of the big ones, believe it or not, is just, I would say, kind of a failure of imagination. And what I mean by that is that you're kind of flash with this, you know, you're presented this blank flashing prompt. And it's like, this is, you know, 200 Nobel Prize winners at your disposal. Go nuts. And what happens is people kind of anchor to the existing mental frames that they know. And that's like Google and chat. So immediately, you start off on chat. So that would be kind of the first challenge, which we kind of help people shift into that agentic mindset. And then you treat it like texting, where it's, you know, one sentence prompts instead of a two hour prompt, which, again, prompting has shifted. But it's not like an iMessage. It's much more like a brief in many of these cases. So I think that's the one of the biggest ones is to kind of like unlock what's possible, right? And so, you know, I joke with people. It's like, you now have a data scientist in your pocket, right? But what does that mean? That means you need to have the data that a data scientist works with. You need to understand the language to communicate with a data scientist. But more importantly, you need to look at your problem space and understand what a data scientist could be useful for. And if you never worked with a data scientist and you might have at some of the firms you have, that is a completely different world. Right. And so I really kind of think of it as the expansion of kind of what's possible. That would be one thing. The second thing is usually just like, how do all the pieces fit together? The data, the prompting, the custom instructions, the memory, the statelessness of an LLM, projects. You just kind of like... So I would kind of put that in kind of like marrying the workflow with the tooling and the features. And by the way, there's like, you know, Anthropic's been on a tear, but they've launched a new two new features a week since January of this year, and we're in April. So that'll be another. And the third is and this is actually a harder one is, I think I saw this quote the other day is, so much knowledge work is just vibes in spreadsheets. And people don't actually know what they do for work. It's the vibes. They're like, okay. You know, it could be anchored around thing like quarterly earnings. Like, there's an... There's a huge...
00:23:02 · Khe Hy, Brett Caughran
They know what they do there. But if you ask them, like, what's your screening process? Like, oh, like, sometimes I look at 13 Fs. Like, we'd go to these idea dinners. There's not, like, an actual... It's Vibes, right? Sourcing happens on Vibes. And this actually isn't an LLM question. This is much more of a process... I don't even... I'm not a... I'm not, like, a formally trained consultant, but it's like process excavation exercise, for lack of a better term. [00:23:34] S0: Yeah. [00:23:34] S1: Or maybe workflow mapping, right? And I think that, you know, it's okay to run with Vibes when, you know, without LLMs because you can kind of, like, flow back and forth and you can kind of work in that kind of osmosis, a little bit reactionary way of doing things. But with LLMs, you need to be clear on your intent. You need to be clear on your steps. You really need to, like, deconstruct things. And that is that's the hardest thing with my clients because every analyst is every firm is different and every analyst is different. And mostly, you know, that is kind of it's a very kind of human exercise of, like, extracting out knowledge and then remapping it towards an LLM.