"The Cognitive Revolution"
00:54:43 · Bryan Bischof
[00:54:43] S4: Let me tell you a story about Blue Bottle. When I was at Blue Bottle, we wanted to build inventory forecasting. And, specifically, we wanted to build a machine learning system that could help us predict how many pastries we were gonna sell on a given day and other things like coffee and milk and so on. But pastries was the first focus. And what was our key, like, north star, our guiding light? Well, it was actually... If this is successful, it's not that we sell a lot more pastries or that we waste a lot fewer pastries. Okay? Those were, you know, two metrics that we tracked. But what would we have believed with success? What we would have believed as success is if the cafe manager who's already got about 70 jobs, if that cafe manager essentially goes into the system and when it comes time to order, they just click okay. Okay. Okay. Okay. Okay. Okay. Now we don't want them to just not think and just autopilot. We actually don't want that at all. And there's a reason. And this will surprise you, but the variance in sort of what a cafe lead understands with respect to their cafe demand is shocking. It is... It would it would blow your hair back how much they understand about the demand in their cafe that is not just in the numbers. We have these weirdly deep, like, you know, tacit understandings. So what was our north star? Our north star was that a cafe manager has more opportunity to spend that time with customers. Because a cafe manager is not superhuman at looking at data and saying, like, oh, I'm gonna need, like, 25 cookies. No. A cafe manager is superhuman at understanding hospitality. And that may sound weird because because you can't manage that on the dashboard. You can't look at hospitality on the dashboard. But what you can do is you can ask them periodically, how much more time do you have available to spend with customers, to spend on the floor and notice that there's, you know, a coffee spill, to understand that your baristas are slammed right now and actually, like, staffing was off. Also my fault. But, and so these are the things that we always cared about. Why do I bring this up? Well, yeah, you could say, like, okay. It's, like, seven to fourteen years to find out if this, like, you know, diligence was good. Sure. But it's a pipeline. So first of all, you wanna ask yourself, how fast can you get through the research pipeline?
00:57:25 · Bryan Bischof
How fast can you go from identification of a potential opportunity or market to a clear understanding to be able to make a decision? But more importantly, hospitality. What does hospitality mean in venture? It means that the investor has time to talk to human beings, to talk to potential or existing companies, to talk to founders and competitors and employees, to try to understand the human element of this market and what's going on. Investors are superhuman at a few things, but I don't personally think investors are superhuman at looking at websites. But they are superhuman at sort of understanding potential, understanding upside, understanding sort of the relationship between, like, you know, customer need, willingness to buy, and sort of framing what the vision in the future can look like. That's what they're superhuman for. And so when you ask me, like, how do I measure success? Well, I measure success when an investor tells me, yeah. Actually, I get to spend 30% less time staring at, you know, the website and trying to figure out, like, the important details I look for every single time. Or I get to spend 20% less time filling out, like, you know, these specific answers to questions which are important for diligence, and instead picking up the phone five more times and talking to five more people who have credible opinions on this deal. That's success. And you don't need to wait seven years to find out.
00:59:03 · Prakash Narayanan
[00:59:03] S1: Do you find there is kind of a spectrum of, like, let's do more research to... We need to cut off the research and we need to make a decision now? Because that research process doesn't really have a defined endpoint. You could always dig deeper, correct? Totally. How do you make those decisions? Are the agents the ones making the decisions? Is there a stepping off point where they have to come back And you make that... A human is making the decision. How does this loop work?
00:59:32 · Bryan Bischof
[00:59:32] S4: Totally. I wanna reference something that Nathan said earlier. He's got this list of things, and he's like, some of this should have been left in the cutting room floor. We would we would call that taste and sort of, like, knowing what should be cut and what shouldn't be cut is taste. And yet again, here's where I will point to why it's important to have exceedingly bright colleagues. When you have exceedingly bright colleagues, they've got great taste on, like, what research needs to get done? There is some research that, like, yes. Like, we always wanna do it. We know we need to do it. This is the structure. And there are opportunities for agents to say, let's spin off some sub agents to continue doing this kind of research. Let's dig deeper on this. That's fine. That's automated. But ultimately, the alpha is in the the taste. And so, yeah, like, we can burn more, you know, tokens. We can burn more compute. But I don't know. It's a pretty it's a pretty, like, focused laser where you really wanna point these resources at things that people have the ability to consume. I... If I produce an extra thousand pages of research on some deal, who's gonna look at it? I guess other agents could look at it, but, like, then what? Is that valuable? It's not obvious to me that it is. And so, again, pure volume of research is not a metric that we even track.
01:00:53 · Prakash Narayanan
[01:00:53] S1: How goes your teaching of data science at I believe Rutgers? Yeah. What do students... Students are probably asking some similar questions there. So what do you say to them and what do you think the future of that field is?
01:01:08 · Bryan Bischof
[01:01:08] S4: Yeah. Great question. I still make them do gradient descent by hand. So this is an area where, in my opinion, I have yet to hear a single person with a cogent explanation of what the hell we're supposed to do as teachers. I believe and have believed for a long time. You know, I came up in pure mathematics. We're we're not the most practical bunch, But I have always believed, even as a pure mathematician, that you should be constantly thinking about what your student's future looks like and why every minute in your class is relevant to that future. I am, not just, like, phobic, but, like, allergic to wasting students' time. So what I do is I reassess every semester, much to my own suffering, because that means I have to rewrite my curriculum every single semester on what do I teach them. So three years ago when it was obvious that sort of AI was going to change data science, and then I started working at Hex trying to change data science, I rewrote my entire curriculum to introduce why AI might be relevant. And every semester, I go through the whole course, and I ask, can AI just one shot the course? The answer has never been yes, but that's okay. The answer has always been, okay. These things are now not just easy for the model, but easy for the model to obfuscate the learning. And so I don't think the question is, does the model know all the content that I plan to teach? The question is instead, does the model have the ability to collaborate with the student when this is gonna be relevant for the student in a way that yields the sort of best outcome. If the model can... If if, you know, Astra can train an SVM on the coffee survey data at a better loss rate than Brian can meet. But can Astra say, you know, this feature is highly predictive, but when we think about it, if we put this on the website, it's going to upset some of the people going through the flow, and they might churn. Like, yes, Astra can look at the data posterior and say, people were churning more on this question. But does Astra know ahead of time to look for that? Does it know to instrument that in the product? Does it know that this is kind of a weird question and people might be bothered? These are the kind of things that Astra Ultra is still only mid at. But, ultimately, I don't want my students to only be users of AI.