Invest Like the Best with Patrick O'Shaughnessy
00:50:17 · Noah Shinn
[00:50:17] S2: One thing that I think is to our advantage is we figure out how to serve the product no matter how you evaluate, whether it's AB tests, whether it's internal evaluations, or it's tracking engagement across users. We're able to deliver the same performance as Opus five, which is now, I guess, we're dating ourselves frontier level intelligence, the same engagement rate, the same AB test performance. It's the same internal evaluation performance, but at a cost that is very low. It is actually very affordable. We're running this program. Every user has the product for free, and our goal is really to deliver this product at an affordable... I'm not gonna commit to free for a lifetime for now, but that is my personal goal to be able to deliver this product for free for everyone for a lifetime. It's just hard infrastructure work. I don't know how deep we wanna get on this, but just if I give one example, if you use Frontier APIs from some of the main providers, you are taking a blanket cost on a certain request and for all of the requests that might be needed to power that product for that month. But there's a lot of different work that's happening through... For activity that is happening throughout the day that doesn't actually need to finish in hundreds of milliseconds and needs to finish in minutes or even hours. There is batch work that is consuming a lot of content that can be served with deployment shapes that are three x, five x, eight x more efficient with the same underlying compute. So I think that when you customize these inference deployments to be able to perfectly shape the data and the workloads that you're serving, you're able to find these 30% here, five x there, six x there, 10% here, and all of those compound to a rate where we're able to serve at a very low cost.
00:51:58 · Patrick O'Shaughnessy
[00:51:58] S1: How do you think about solving the bigger problem of how far ahead to buy? And if I think about this at true scale, like you get to scale of a billion users or something like this, how much new compute demand do you think this will represent? I mean, it seems... Cogent's obviously been an enormous amount. But ground us in some sense of scale, whether it's per user or some way of understanding how much new compute this will require.
00:52:20 · Noah Shinn
[00:52:20] S2: It's going to be a lot. Let's just ignore compute and just think about how many tokens are flowing through the platform. The products that are breakout products of a couple months ago, let's say in the cogeneration space where it requires the user to prompt it and then it goes and runs for something and then it comes back to user and then asks for something else and then the user sends something again. Sure, a lot of that is background work, and so there's quite a bit of tokens that are consumed there. But here, Instinct has the ability to wake up and to sleep at any moment in time during the day. That might sound a little odd, but its architecture is enabling it to do that so that it can really think about, you know, if you have a meeting that you're running late to and you need to order an Uber or if they ordered the Uber and the user is not showing up, I'll be able to help the user through those moments. So productivity, I think, is going to continue to expand over time. There's this big compute build out with the earlier breakout products in the AI space that are just scratching the surface of productivity or a background work. Here now, we have something that is almost natively proactive. It is a smaller subset that is actually interactive. So I think the amount of compute that is going to be needed is going to be orders of magnitude more than what we thought that we needed. It's just with more productivity comes more care for the user, more time to think about certain things that could go wrong or not go wrong. There's just so much that's happening under the hood. Maybe instinct wakes up at 6AM because it knows that you wake up at 7AM, and then it goes and scans everything, make sure that everything's ready for the day for you. And then it realizes, hey. Actually, now is not a great time. Like, it should just go to sleep or do this one thing in the background and not notify the user. And then it realizes, oh, at 4PM, there's something that's coming up. There's value that could be provided. Maybe the user doesn't even know how to interact with Instinct in that way, but Instinct thinks that it's well meaning and it's something valuable to the user. So it might then just wake up at at 4PM and then do the task and contact the user, and then the user could lean into it and do that task.
00:54:11 · Noah Shinn, Patrick O'Shaughnessy
You see how much work is going on in the background with these coding products? You don't really have that much, you know, background or productivity work happening. I don't mean to quote a number here. I'm just saying that the shape of the product and of the workload that's going to be run, I think we're just scratching the surface in terms [00:54:27] S1: of how many tokens. What do you think of Muse? What do you think of the product?
00:54:27 · Patrick O'Shaughnessy, Noah Shinn
[00:54:27] S1: of how many tokens. What do you think of Muse? What do you think of the product? [00:54:30] S2: I think it's a great product. I was playing around with it for a little bit, and it's interesting. It's a different take. Sure. You can say some of the underlying architecture might be similar, but I think it's fundamentally different. I think that instinct is meant to be simple and very easily accessible. The soft qualities that we were talking about earlier of really thinking from the user standpoint about what's important and what's not important and how to make this task easier and easier to read and other things like that versus a new application and a new interface. Sure, maybe over time, we have an application that also delivers on a different set of tasks, but I think it's a great product. I just spend very little of my time thinking about competition and other players in the space because you walk outside, you go to the nearby cafe, and you think about how many people within that cafe are actually using AI in the way that they imagined that they would or the way that they want to be using it, and I would say very little. And that's here. Right? You go over to other countries or other cities, and it's certainly lesser of the case. So it's still an open space. It's still an exciting time. I think it's an interesting game that's going to be played and rolled out over the next coming months and years. I'm just focused on building [00:55:37] S1: the best product experience.
00:55:37 · Patrick O'Shaughnessy, S0
[00:55:37] S1: the best product experience. [00:55:39] S0: Your finance team isn't losing money on big mistakes. It's leaking through a thousand tiny decisions nobody's watching. Ramp puts guardrails on spending before it happens. Real time limits, automatic rules, zero firefighting. Try it at ramp.com/invest. As your business grows, Vanta scales with you, automating compliance and giving you a single source of truth for security and risk. Learn more at vanta.com/invest. Bridgeline offers one unified platform that automates away the complexity across portfolio accounting, reconciliation, reporting, trading, compliance, and more, all at scale. Schedule a demo at ridgeline.ai. Every investment firm is unique, and generic AI chatbots don't understand your process. Rogo is built by finance professionals. It's connected to your data, learns your process, and quickly produces client ready outputs. Check them out at rogo.com/invest. The best AI and software companies from Open to cursor to perplexity use WorkOS to become enterprise ready overnight, not in months. Visit workos.com to skip the unglamorous infrastructure work and focus on your product. [00:56:48] S1: What have been the blunders so far? Like, what's gone wrong? What have you done about it? This is just gonna be an explosion of emergent properties and mistakes, and the end's gonna be really high. How do you think about things to guard against proactively, things to be reactive against? Like, talk us through the darker side of building this or the harder side of building this.