Business Breakdowns
· David Plon
[00:00:00] S1: Obviously, everything we spoke about earlier with experimentation certainly applies here. But I think one thing that is going to be really important, and already was historically, but I think carries a lot more weight going forward, is just the importance of documentation of thinking and decisions and research. Essentially, these models, the usefulness of them rises exponentially with the amount of context they're given. As the model context length expands and as their agentic reasoning gets more heavily utilized to do longer-running tasks, and they ultimately shift from being a helpful research tool to something that lives within a firm and can execute a research process, their ability to do it is going to be a function of how much data they have on how you and your firm operates. So I think firms where they have documented memos and even write up short paragraphs on why they made a trading decision – it's hard to know ex-ante which data is going to be used and how – but I think it's a pretty reasonable bet that having that data, at a minimum, is helpful for the humans, but will certainly be helpful for the machines. Because I think – a moment ago we were speaking about adoption – what’s challenging is loading in that context. It's the same challenge it goes when you hire a junior analyst: needing to teach them everything about how you think and your past trading history with a given name. And all that information just takes a ton of time to convey. And so to the extent that that data can be captured in real-time, that forms an enormously valuable source of IP that will make AI unique to you. And again, it's a little hard to know upfront how that's going to be used, and it probably takes some creativity to figure it out. But hard to imagine two or three years from now that every piece of data isn't being used within a model that is operating within the context of a fund.
· Matt Reustle
[00:00:00] S0: It's a perfect example. I could speak to on-ramping at places where all they had were the memos, which were the buy and sell decisions for positions, which are valuable in some ways, but they're usually overly polished, versus those that after earnings they would have their quick blurb. Maybe Amazon entered the space and was something to monitor, and then a month later they had a meeting with the management team and it made them feel a little worse off, and then another quarter comes out and it's clear Amazon's impacting them, and the memo might just say "Amazon's movement." But to see the thought process and how the team makes decisions, there was something very valuable in me getting up to speed. Again, I think it goes back to that lens of almost thinking of it like a team member in the same way that you would give them the most ammo to align with how you work and what you do. Going back to some of the quick-hitting questions, thinking about down the road and where we're going. I already brought up context windows. We'll use LLMs again, just thinking about some of the most used tools. When you think about where investors – or where you personally find the context window advantage has the most impact, or how you adjust for the challenge that I think many investors face, which is I want to get as much information in there as possible, but sometimes I'm limited by it. How do you approach that, and how much do you think you should worry about that as an investor?
· David Plon
[00:00:00] S1: It's super dependent on what tool you're using, and in what context you're using it, and the task. To go a little deeper on that, the context windows obviously have been expanding, although really in the past year they've been flat, with Gemini at around a million, GPT I think around 400,000, and Opus or Claude around 200. Where the models have gotten more capable is using that context well. I think historically models struggled if you loaded in, I don't know, the last five 10-Ks. And let's say a 10-K is like 100,000 tokens, so 500,000. The models historically would be really good at say if you say, "What was SG&A in 2023?" They call that a needle-in-a-haystack problem. They would struggle if you said, "Build a three-statement model using these." The model needs to apply its attention across lots of different pieces of the context window simultaneously. I think that still is a roughly helpful rule of thumb, which is the models are generally, if you're using a ton of context, you can do better if you're loading that in because you're looking for very specific data points. That being said, I think a change versus if we had this conversation three months ago is – models have gotten very good at using the context intelligently. If you're thinking about building a project in Portrait or NotebookLM or something like that, I wouldn't really hold back. I would give it pretty much the entire corpus. And you're seeing this a lot in software engineering, where Claude Code and other agentic coding tools are very capably exploring super long context in a way that is efficient and focused on the task at hand. So in the same way that I wouldn't give a junior analyst a task and say, "These are the only four documents you can use." You'd want them to ultimately have the agency to choose what content they're going to use. And I think that holds true. All that being said, if you're going to use the raw model itself, you do not want to overload the context window and use 70% of the context. You will see a degradation in any complex task. In that case, you should break it down. But hopefully the much easier way to do this is use a tool that manages that complexity already, so you don't need to worry about, "Okay, what percent of the context window am I utilizing?" and so forth, is how I think about that today.
· Matt Reustle
[00:00:00] S0: On memory, I think it's obvious where, as memory has now been implemented in a lot of these tools, it customizes to you in that iteration. Again, I'm going to overuse the lens of the employee overseas, that you're creating a better, more aligned working environment. Are there any non-obvious impacts that you think memory will have?
· David Plon
[00:00:00] S1: I think the interesting thing about memory is – today, as it's implemented, it's far worse than a human's memory. It is super limited. It's expensive in the sense of having deep memories takes up a lot of context. It lacks a lot of the nuance of – our memories are really useful, both for specific facts, but more so for the abstract concepts that they create and that we learn. When you think of those memories like, a lot of times that connection is where they become really powerful, and that's become pattern recognition for investments. So in the near term, I think memory is a useful tool for all the obvious reasons that you can think about, in terms of just adding more context that persists in the context window. Longer term, everyone knows all the fallibility of human memory. And this is a little bit above my pay grade in terms of the actual ML research and how memory is being incorporated in different ways. But you could imagine any one of the architectures that are being experimented today with memory, and having short-term memory, and longer-term memory, and the ability to look up memories. If implemented correctly, both in terms of how the models are trained as well as the harnesses around them, imagine having a model that has lived every single one of a firm's investments and maybe things they even passed on. There's a world where it becomes so much more powerful than any given human. I look at some of the investors I've worked for, and they just have this insanely, eerily intuitive judgment about things. A lot of that is not just memory of specific facts, but memory that comes from experience and scar tissue. There's a world where that certainly could exist in models. And I think in some sense, people talk about AGI being the ability to be generally intelligent. I think a really interesting test for a model, if it’s AGI is – can it predict the future? You need a really good world model and understanding of lots of variables to predict the future, and that's what an investor does. And memory is clearly a really important ingredient to that. So I'd say near term, I'm a little bearish on just how big of an impact it's going to be relative to just copy-pasting that context into a window.
· David Plon, Matt Reustle
But longer term, I think that's the unlock where this goes from being the useful tool to the real core driver of a lot of the research a firm is doing. [00:00:00] S0: Well, I'll separate it. Near term, I'm almost taking a way that it might be more valuable to limit the memory, therefore reducing the context window, and investing more into the prompt that I'm using. I know they're not like-for-like, but ultimately memory is customizing to you, and if you could just put that in the prompt, that might be more beneficial than sucking up memory.