The Frontier with Brendan Foody
00:31:54 · Jonathan Frankle
An agent does and will find something it should not have found, HR-related, legal-related, what have you, that may actually cause some trouble for your business. Like the permissions today are probably not very rigorous as it stands. And then you as a human need to understand what permissions did I give my agent and what could an agent do with those permissions that I would never have thought of. I had a fun example at Databricks where you know I gave an agent access to Jira and a Google Doc. It's like, oh, what could go wrong there? And actually narrowly avoided a really complicated scenario because I didn't think about what data was in both places and what would happen if you combine those two permissions together. Now multiply that across Slack, you know, glean, what have you. And that's just services, not individual Google Docs that you might or might not have access to. Things get messy really quickly. And I think the evolution for evals is not just about did the task get completed, but what was the process? And was that process a good and reasonable process? What permissions did the agent need? What did it access? Should it have accessed that? Does it need internet access? I gave it internet access. There's no like you just have permission to view this one website. There's no read and write permissions for the internet. There's just get requests and post requests. You can write data to the internet really easily without having permission to do that and weird stuff happens.
00:33:04 · Brendan Foody
[00:33:04] S1: It feels like so much of this has come from the fact that agents have improved so much over the last 12 months that enterprises trust them to do more or at least want to. So as we give agents more autonomy to interact with access context, to use tools, that also comes with the risk associated with them and how we build the right guardrails. I'm curious, how has progress of models and agents over the last 12 months compared to your expectations?
00:33:31 · Jonathan Frankle
[00:33:31] S0: I think it's been ahead of my expectations in general, but I tend to always be a little more pessimistic. You ask my friends three years ago. I have a friend who's now at Anthropic where I think he told me two years ago that he thought that by 2026 or 2027, AI was going to be able to address like 50% of all GitHub issues out there. It's like, you're an idiot. It's going to be like 5%. These are really hard tasks. He's totally right. I was a little pessimistic. The way I look at it is more of, you know, this is going to sound dumb, but like roll back the clock to December of 2025. I know it's a very long time ago at this point, and ask yourself, what are the like, what did all the VCs say the big trends in AI were going to be in 2026? And it was things like we're going to deploy agents widely for work, memory is going to be a really important topic, open claw was a hot topic. Like, we're really going to delegate our whole lives to agents. Coding agents were going to be really important. It was just this unbridled optimism about everything exciting that was going to happen. And I was sitting there, and you were sitting there saying the same thing of like quality is going to matter and measuring quality is going to matter, like evals, evals, evals, and evals were going to be the big trend. When I talked to my customers today, I would say the three big trends I hear most or the questions they ask me first are cost. We were in a three-month period of token maxing and leaderboards. And now we're, God, I can't believe I used this word in front of 1,500 people. But I said the era of token maxing is over and the era of value mogging is here. And I'm I'm going to cringe, but I said it anyway. Again, recorded again. Um, the second one is governance. What should an agent have access to? What shouldn't it? And how do you avoid an agent doing really stupid, inexplicable things that actually cause you a lot of risk? And the third is security. I mean, the fact that now we don't even know what code is being written and an agent can pip install anything. And we've seen the number of compromises in pip installations, like that's actually really scaring enterprises. And I'm seeing it cause them to shy away from even using software from startups because you don't even know what to do.
00:35:20 · Jonathan Frankle, Brendan Foody
So the all this is to say that like it's very interesting that we've gone from this very optimistic posture, kind of as a field, as an industry, and as business in general. I feel like everybody's kind of like battening down the hatches and getting very afraid. And I hate that in some sense. Like I feel like there's an in-between we can live in where it's been an interesting spring. Like some stuff has happened. It's been hard and complicated and we've all made some mistakes. [00:35:42] S1: Yeah, I definitely feel like in the first four or five months of this year, it was all about token maxing. It was this first phase of deployment of how do we just plug the models into the context and the tools and see what they can do. And then as Athropic and OpenAI revenue skyrocketed, naturally the question became how do we optimize this? How do we find the proto-frontier of cost and performance? And how do we understand whether we actually need to be using the largest model to accomplish this task? And obviously, the foundation for all of that optimization is the Eval, which a lot of it is flowing back to. So that makes a lot of sense. But I think as models improve so much in the economy, it's also begging this important question of what role will humans play. I'm curious how you think about that and the role that humans will play in the enterprise uh over the next five to 10 years.
00:36:34 · Jonathan Frankle
[00:36:34] S0: Yeah, I think this is one of those things where people have these really oversimplified views of like all the humans will be replaced, or none of the humans will be replaced, or that, you know, entire job functions will go away. We won't have any more, I don't know, I'm picking on lawyers. I can say this because my parents are both lawyers. We'll get rid of all the lawyers or something. Not only is that like an oversimplified view that's wrong, but it's also not helpful to tell that story. Like the idea that we're going around telling that story is wrong. And like I can understand if someone says, yes, we're going to replace all the lawyers, I can understand why a lawyer might say, well, let's not have any more data centers. I think this is, you know, if you take a very extreme oversimplified position, I can understand why someone would oppose that position. I think we're actually headed for a nice Goldilocks scenario where work is complicated. The more time I spend with my customers, the more I learn that like there's no such thing as AI for finance. Finance is thousands of different individual little tasks, each of which is both complicated and like nuanced and also different at every single organization because you have all the legacy of how your business operates and how it's structured. And there may be three humans in the world, if that, who understand the details of any one task. The phrase I like to use is like the world is fractal in complexity. You look at any problem and you see the same level of complexity as you look more closely and more closely and more closely. So the idea of a finance benchmark or a biology benchmark or a medical benchmark is just to me so absurd and honestly disrespectful to the folks who are actually doing that work every day. And I think what's likely to happen is as with digital computers, as with calculators, as with the internet, work is going to change because there's a lot of stuff we don't need to do anymore. You no longer need to pull out your typewriter and typeset something, copy, edit it, and then retype it. That's gone. We no longer need to have someone who listens to audio tapes and transcribes them. That's gone. That's really boring work that nobody wants to do.
00:38:28 · Jonathan Frankle, Brendan Foody
And instead, the task of doing law no longer requires a typewriter and document preparation in the like word processing sense is no longer an issue. Instead, people are doing more law. So I think it's going to push humans both up a level, but it's also going to push humans to the boundary of tasks. If the really simple, obvious stuff in any domain is relatively automatable, that doesn't mean we no longer need you any more than I think you were just mentioning to me, like, cool, we've like agriculture has become efficient. So now 95% of humans are out of work because they used to all work on agriculture. No, we've pushed to the boundaries of more interesting, more sophisticated things. A lawyer who can spend their time on the most nuanced, challenging issues, or who can help a lot more people. We're not in a world where we have an abundance of legal counsel. How many people in the world, do you have an estate plan right now? Not yet. And you're the kind of person who should definitely have an estate plan, but it takes a while. [00:39:20] S1: Yeah. [00:39:21] S0: Like what percent of people have an estate plan, and yet we're all going to need an estate plan someday. [00:39:27] S1: Yeah. [00:39:27] S0: That is one universal human thing. Yeah. We're all going to need an estate plan someday. And I say that my dad's an estate lawyer, so this is you know near and dear to my heart. But every single person is going to need one and it's expensive and complicated. Wouldn't it be great if everyone could have one?