The Frontier with Brendan Foody
00:36:34 · Jonathan Frankle
[00:36:34] S0: Yeah, I think this is one of those things where people have these really oversimplified views of like all the humans will be replaced, or none of the humans will be replaced, or that, you know, entire job functions will go away. We won't have any more, I don't know, I'm picking on lawyers. I can say this because my parents are both lawyers. We'll get rid of all the lawyers or something. Not only is that like an oversimplified view that's wrong, but it's also not helpful to tell that story. Like the idea that we're going around telling that story is wrong. And like I can understand if someone says, yes, we're going to replace all the lawyers, I can understand why a lawyer might say, well, let's not have any more data centers. I think this is, you know, if you take a very extreme oversimplified position, I can understand why someone would oppose that position. I think we're actually headed for a nice Goldilocks scenario where work is complicated. The more time I spend with my customers, the more I learn that like there's no such thing as AI for finance. Finance is thousands of different individual little tasks, each of which is both complicated and like nuanced and also different at every single organization because you have all the legacy of how your business operates and how it's structured. And there may be three humans in the world, if that, who understand the details of any one task. The phrase I like to use is like the world is fractal in complexity. You look at any problem and you see the same level of complexity as you look more closely and more closely and more closely. So the idea of a finance benchmark or a biology benchmark or a medical benchmark is just to me so absurd and honestly disrespectful to the folks who are actually doing that work every day. And I think what's likely to happen is as with digital computers, as with calculators, as with the internet, work is going to change because there's a lot of stuff we don't need to do anymore. You no longer need to pull out your typewriter and typeset something, copy, edit it, and then retype it. That's gone. We no longer need to have someone who listens to audio tapes and transcribes them. That's gone. That's really boring work that nobody wants to do.
00:38:28 · Jonathan Frankle, Brendan Foody
And instead, the task of doing law no longer requires a typewriter and document preparation in the like word processing sense is no longer an issue. Instead, people are doing more law. So I think it's going to push humans both up a level, but it's also going to push humans to the boundary of tasks. If the really simple, obvious stuff in any domain is relatively automatable, that doesn't mean we no longer need you any more than I think you were just mentioning to me, like, cool, we've like agriculture has become efficient. So now 95% of humans are out of work because they used to all work on agriculture. No, we've pushed to the boundaries of more interesting, more sophisticated things. A lawyer who can spend their time on the most nuanced, challenging issues, or who can help a lot more people. We're not in a world where we have an abundance of legal counsel. How many people in the world, do you have an estate plan right now? Not yet. And you're the kind of person who should definitely have an estate plan, but it takes a while. [00:39:20] S1: Yeah. [00:39:21] S0: Like what percent of people have an estate plan, and yet we're all going to need an estate plan someday. [00:39:27] S1: Yeah. [00:39:27] S0: That is one universal human thing. Yeah. We're all going to need an estate plan someday. And I say that my dad's an estate lawyer, so this is you know near and dear to my heart. But every single person is going to need one and it's expensive and complicated. Wouldn't it be great if everyone could have one?
00:39:27 · Jonathan Frankle, Brendan Foody
[00:39:27] S0: That is one universal human thing. Yeah. We're all going to need an estate plan someday. And I say that my dad's an estate lawyer, so this is you know near and dear to my heart. But every single person is going to need one and it's expensive and complicated. Wouldn't it be great if everyone could have one? [00:39:40] S1: If we were to itemize all of your customers' problems and even the broader economy into tasks, those tasks obviously require reasoning, but more importantly, they also require context. Contextual to that business, contextual to that problem, et cetera. What percent of those tasks do you think that the models have actually automated so far?
00:39:40 · Brendan Foody, Jonathan Frankle
[00:39:40] S1: If we were to itemize all of your customers' problems and even the broader economy into tasks, those tasks obviously require reasoning, but more importantly, they also require context. Contextual to that business, contextual to that problem, et cetera. What percent of those tasks do you think that the models have actually automated so far? [00:40:00] S0: Very, very, very little. I would say we're in the single-digit percents and probably the very low single-digit percents. I think some of it may be very difficult to automate for a really long time. The fundamental breakthrough we need, in my view, is not more intelligent models, but more cost-effective e-file development. And cost effective, not in the sense of like you spend less money, but that it is for the three humans who know how to do this one tiny enterprise task, it's not miserable for them to do this. And they feel like maybe it's even delightful. And maybe it's not just an extractive task where they feel like they are getting the knowledge pulled out of them into an agent, but they are growing and learning and they become more sophisticated, what they want to do. This is kind of, you know, in some sense, the EM algorithm in AI, you know, you, you know, expectation maximization. It's a back and forth process where things get better and where, you know, the idea of building an eval is also a learning process if you structure it that way, if you can make it delightful. And to me, that is the ultimate place where we need a breakthrough right now. If we can do that, we can unleash AI and unleash humans.
00:40:00 · Jonathan Frankle, Brendan Foody
[00:40:00] S0: Very, very, very little. I would say we're in the single-digit percents and probably the very low single-digit percents. I think some of it may be very difficult to automate for a really long time. The fundamental breakthrough we need, in my view, is not more intelligent models, but more cost-effective e-file development. And cost effective, not in the sense of like you spend less money, but that it is for the three humans who know how to do this one tiny enterprise task, it's not miserable for them to do this. And they feel like maybe it's even delightful. And maybe it's not just an extractive task where they feel like they are getting the knowledge pulled out of them into an agent, but they are growing and learning and they become more sophisticated, what they want to do. This is kind of, you know, in some sense, the EM algorithm in AI, you know, you, you know, expectation maximization. It's a back and forth process where things get better and where, you know, the idea of building an eval is also a learning process if you structure it that way, if you can make it delightful. And to me, that is the ultimate place where we need a breakthrough right now. If we can do that, we can unleash AI and unleash humans. [00:40:57] S1: Yeah. [00:40:58] S0: I don't think this is mutually exclusive any more than, you know, to what you told me earlier, like making agriculture efficient did unleash humans. We had more food, we didn't need as many people to do manual labor. And instead, we built our modern world.
00:40:58 · Jonathan Frankle, Brendan Foody
[00:40:58] S0: I don't think this is mutually exclusive any more than, you know, to what you told me earlier, like making agriculture efficient did unleash humans. We had more food, we didn't need as many people to do manual labor. And instead, we built our modern world. [00:41:12] S1: What do you see that work looking like in five years? Like, do you think that people spend a lot of time building evals, updating the context of agents, uh just operating at higher levels of abstraction, managing those agents? What will that look like as we think about this category for training agents and how it develops?