"The Cognitive Revolution"
00:59:03 · Prakash Narayanan
[00:59:03] S1: Do you find there is kind of a spectrum of, like, let's do more research to... We need to cut off the research and we need to make a decision now? Because that research process doesn't really have a defined endpoint. You could always dig deeper, correct? Totally. How do you make those decisions? Are the agents the ones making the decisions? Is there a stepping off point where they have to come back And you make that... A human is making the decision. How does this loop work?
00:59:32 · Bryan Bischof
[00:59:32] S4: Totally. I wanna reference something that Nathan said earlier. He's got this list of things, and he's like, some of this should have been left in the cutting room floor. We would we would call that taste and sort of, like, knowing what should be cut and what shouldn't be cut is taste. And yet again, here's where I will point to why it's important to have exceedingly bright colleagues. When you have exceedingly bright colleagues, they've got great taste on, like, what research needs to get done? There is some research that, like, yes. Like, we always wanna do it. We know we need to do it. This is the structure. And there are opportunities for agents to say, let's spin off some sub agents to continue doing this kind of research. Let's dig deeper on this. That's fine. That's automated. But ultimately, the alpha is in the the taste. And so, yeah, like, we can burn more, you know, tokens. We can burn more compute. But I don't know. It's a pretty it's a pretty, like, focused laser where you really wanna point these resources at things that people have the ability to consume. I... If I produce an extra thousand pages of research on some deal, who's gonna look at it? I guess other agents could look at it, but, like, then what? Is that valuable? It's not obvious to me that it is. And so, again, pure volume of research is not a metric that we even track.
01:00:53 · Prakash Narayanan
[01:00:53] S1: How goes your teaching of data science at I believe Rutgers? Yeah. What do students... Students are probably asking some similar questions there. So what do you say to them and what do you think the future of that field is?
01:01:08 · Bryan Bischof
[01:01:08] S4: Yeah. Great question. I still make them do gradient descent by hand. So this is an area where, in my opinion, I have yet to hear a single person with a cogent explanation of what the hell we're supposed to do as teachers. I believe and have believed for a long time. You know, I came up in pure mathematics. We're we're not the most practical bunch, But I have always believed, even as a pure mathematician, that you should be constantly thinking about what your student's future looks like and why every minute in your class is relevant to that future. I am, not just, like, phobic, but, like, allergic to wasting students' time. So what I do is I reassess every semester, much to my own suffering, because that means I have to rewrite my curriculum every single semester on what do I teach them. So three years ago when it was obvious that sort of AI was going to change data science, and then I started working at Hex trying to change data science, I rewrote my entire curriculum to introduce why AI might be relevant. And every semester, I go through the whole course, and I ask, can AI just one shot the course? The answer has never been yes, but that's okay. The answer has always been, okay. These things are now not just easy for the model, but easy for the model to obfuscate the learning. And so I don't think the question is, does the model know all the content that I plan to teach? The question is instead, does the model have the ability to collaborate with the student when this is gonna be relevant for the student in a way that yields the sort of best outcome. If the model can... If if, you know, Astra can train an SVM on the coffee survey data at a better loss rate than Brian can meet. But can Astra say, you know, this feature is highly predictive, but when we think about it, if we put this on the website, it's going to upset some of the people going through the flow, and they might churn. Like, yes, Astra can look at the data posterior and say, people were churning more on this question. But does Astra know ahead of time to look for that? Does it know to instrument that in the product? Does it know that this is kind of a weird question and people might be bothered? These are the kind of things that Astra Ultra is still only mid at. But, ultimately, I don't want my students to only be users of AI.
01:03:48 · Bryan Bischof
I want them to be sort of the things that make data scientists exceptional, and that is ultimately curiosity. Like, every data scientist that I've ever met that is in the top decile and certainly in the top percentile are curious as hell, almost to a fault. The absolute best data scientists I've ever worked with are borderline annoying with their curiosity. Like, you kinda want them to just, like, leave it alone sometimes, and that's what makes them so excellent. And so mostly, what I'm trying to, like, indoctrinate my students into the propaganda is curiosity. Be curious. Ask the questions that you can't get out of that brain. And when you do that, you're gonna be an excellent data scientist. Whether you're writing PANDAS by hand, whether you're using matplotlib from memory, I don't really care. Do you remember the indexing for NumPy? I don't. I used to know it. I don't remember it anymore. Do I remember the slice operators for pandas filters? Not anymore. I forgot them all, but I used to know them. It doesn't matter, though. What I know now is I know sort of when you look at a distribution, whether it makes sense, and sometimes Astra doesn't.
01:05:10 · Nathan Labenz
[01:05:10] S0: Part six, which hills? After Bryan Bischof signed off, Prakash and I kept going. That, his answer on what he tells his future student or his students about their futures, I thought was really interesting, and it it sort of contained the pessimist and the optimist take at once when he was first going through the... You know, sometimes there's, like, survey question that may offend people, and you gotta be wise to that. I was like, man, that's bleak. You know, the idea that, like, we're just gonna be sitting here being the sort of, offense detectors, you know, while the AIs do all the real work. I was like, oh god. You know, that doesn't sound awesome. I did like the curiosity framing a lot better. Being relentlessly curious, certainly in my experience, working with data has been indispensable. You know, really want... The difference between somebody who really deeply wants to know the truth that the data contains to the degree that the data even contains truth, which is often a problem unto itself, those people absolutely kill it compared to those that are just, like, trying to produce an artifact that they can say, you know, is done and and hand off to, whoever the consumer of it is. So I do like curiosity as an answer to, like, what we're gonna bring to the future, but it does. I would not be telling my kid to go into data science today. I'll tell you that. So that is a tough one. I don't... You know, the the real question there of, like, is there going to be a job in data science that looks anything like the current job in the next few years, I I feel like that is... I think that's pretty tough. You know? It seems like we're gonna have data science agents, and it's gonna be up to the sort of non data scientist general purpose business operator to provide that little salt of curiosity or offense detection or whatever that, you know, special sauce is that the models don't quite have.