Business Breakdowns
· David Plon
[00:00:00] S1: In an evolving technology like this, I think spending some 15% of your time on experimentation is really important, because unlike past technologies – one, the underlying models are changing, but two, the capability frontier is evolving as well and is quite jagged, and there are a lot of undiscovered, for lack of a better term, capabilities that you can pick up on well before others do. And so having a process by which folks experiment and really spending some time pushing the models can be one, helpful in its own right, but two, gives you a better intuition of where the models are today, and maybe where they're going. So, to add a little specificity to that, I and we at Portrait, have plenty of tasks like this, and even me personally, I have some tasks where I know the models can't do this today. Or maybe they can do them a small percentage of the time. But anytime there's a new model that comes out, I run my suite of 10 different exercises across it. And one, I can get a feel for how much of a leap this model is, but two, I might now need to update where I think the edge of that capability frontier is. And a lot of times once you start doing that process, it kind of builds on itself, where you might have an experiment – maybe it is building a cost curve and how that's shifting. That's a type of task that can take a long time and be challenging, and now the model can start doing it. Well, great. Now you start having ideas of, "Well, I wonder if I layer in the long-term cost curve versus a marginal cost curve," and there are different things you can start coming up with for ways you could actually apply it within the research process today. So I would say for anybody – certainly companies do this, but individual investors – have a suite of 10 different things where if the models could do it, it would be great, and just constantly run these against the model. And once they are able to do it, save that template and reuse it as part of your research process when working on a name. So I think those are some of the things that where experimentation can pay a lot of dividends.
· Matt Reustle
[00:00:00] S0: I would bucket this in experimentation, but one of the things that you laid out when you talked to Brett from Fundamental Edge, was just writing prompts in different forms. So start out with the most simple prompt that you know is probably going to give you a very broad, mediocre answer. Then take it a layer down in more detail and specify more detail, and then see how the responses change based on how you're instructing it via prompts. I did it a bit, and it made me appreciate how it's sensitive to maybe the ordering of questions or if I specify the ordering of questions and what I'm loading into the context window. And it doesn't just impact that single exercise; it makes you appreciate that this is a technology that really doesn't have many buttons that tell you, "Oh, this is four-wheel drive, that I use in this condition." It is just a chat, and you have to play around with it yourself.
· David Plon
[00:00:00] S1: And just on that, I think one thing that's really cool – I mentioned earlier with the way I write prompts as I think about sending an email to someone overseas. The difference with an LLM is that the response is, for all intents and purposes, instant. The real insight, I think, comes from doing exactly what you just described, which is iterate on this. The cost of sending a single query is trivial. Start simple, start adding complexity as it's helpful. It's really like a two-way dance to end up in a spot where the AI really is acting like a useful analyst. And you don't need to wait overnight to get the content to then see how to give feedback. You can give that feedback real-time and adjust things, and that becomes super, super powerful.
· Matt Reustle
[00:00:00] S0: Your prompts are loaded into Portrait for various primers or whatnot, and you see the layers, the levels of detail that go into those, and it's actually quite helpful. It made me feel a little down on myself in terms of what I was prompting prior to seeing those, but I think, yeah, it definitely rings true. Taking it up a level, I have seen quite a few individuals who are getting a lot of value out of AI, but it almost feels like it's siloed. You could have two individuals at the same fund that use entirely different tools, and they're getting value. Obviously, the enterprise or the fund is benefiting from all of it, but it doesn't necessarily feel strategically aligned, or that there’s too much collaboration in regards to it. Have you noticed anything in terms of best practices for funds just to get some type of adoption that's aligned or collaborative?
· David Plon
[00:00:00] S1: Because investing is ultimately – a lot of the research is building conviction, and people build that in different ways. It's challenging to mandate changes in a research process because if I was at my last job and the head of the firm said, "Okay, now you must use only sell-side models for your models," I would really struggle. I can certainly sympathize with that. I know a lot of firms have been trying top-down to say, "Hey, we don't want to fall behind on AI. You need to start using such and such tool." And I think that can sometimes work in certain ways, but in other ways it can backfire and people push back. What I've seen the most adoption and firms that are taking the most advantage of this, I'd say, is finding the right balance between having firm-wide initiatives that don't force people to change their behavior, while letting individuals experiment and figure out where AI is going to be additive, and reducing friction without negatively impacting conviction. So for instance, just using Portrait as an example, there are firms where, at the firm level, we spend a lot of time building idea generation screens that are super bespoke to them and their process, and we just run those like an outsourced analyst. And that's really value-add, because all we're doing is, we're not asking anyone to change the process. They just get a bunch of really interesting pitches that now they can spend time on if they want. And similarly, the thesis monitoring has become a really big thing. And the nice thing about thesis monitoring is it doesn't require a change in process. All we're doing is we take a ticker and maybe your investment thesis, and then we're just pulling in data points that, at least our system believes, could be really helpful and incremental to your thinking. And again, that doesn't require anything new. It just requires looking at a new insight when it arrives and ignoring it if it's not relevant. Whereas I think for the actual day-to-day asking queries, building outputs, it really just depends on the user. And obviously, working with a firm that specializes in this and has built software specifically for this vertical can be really helpful for driving adoption. Because unlike a general tool like ChatGPT, someone building software in their space can customize the UX around the specific workflows.
· David Plon, Matt Reustle
But ultimately, I think that adoption has to happen bottoms-up, and people need to feel comfortable with it to be able to continue making high-quality investment decisions. And to the extent people have changed the process – and many people have – they've done so because they feel trust in it. And I think building that trust has to happen on the individual level. [00:00:00] S0: Yeah, it's actually an interesting perspective that I somehow hadn't thought about, but when it comes to any type of investment research and investment process, people do it all differently within the same fund. They might have the same core things that they look for, but the path to get there is quite different. And even model formats – I can remember many an argument over having a single specified template and people moving off of that. So yeah, I think there's a lot to that, and maybe that opened my eyes a little bit. On the practices that firms or individuals can do in terms of investing in things that maybe aren't obvious in value today. But as the technology evolves, maybe they'll be able to leverage more. And I mentioned before we talked, there's the famous example of the sports analytics teams that had been tracking data well before sports analytics were a thing, and they were only able to really benefit from all those data practices decades into the future. But the teams that never kept up with it, it's really hard to go back in time. So are there exercises or practices that you think funds or individuals can start to implement that will pay off in time?