00:14:11 · John Daley
[00:14:11] S1: Yeah. So I think, as Jack mentioned earlier, our our... A graph database Atlas, and I think it really comes down to, have you architected your platform assuming it is agents doing most of the work? Right? And I think, like, really foundationally, you need to have that assumption that we are building for a world where it is agents talking to other agents and people to both collect data, do assessment, and then take action. And that's been our foundational belief since we began building the platform, you know, eighteen months ago. I think the... A legacy of platforms who bolted on AI chatbots, sort of... There's a veneer of AI capability, but they're really not providing agents that sort of... That context and that capability in the background to do as much as effectively and as accurately as as what we built. And then there are other AI native platforms out there, but I think they haven't necessarily internalized some of the costs around that kind of data lot of building, which we work on very early in our engagements with customers, and this is informed both by Jack's research and also my time at Palantir where we were very early to see that. That data model of your enterprise is actually, like, the key differentiating thing in many ways. And so we internalize that, and when agents are carrying their tasks, they have the ability to understand like an analyst or a person would. Right? Why am I doing this? Right? Well, it's because I'm in this part of the org. This is the type of concern we have. Here's the data I'm looking at. Right? And I think that's a that's a core thesis we have.
00:15:48 · Sebastian Johnson, John Daley
[00:15:48] S0: And this has become a a particularly hot topic, I think, at the moment in the last few months given some of the incidents that we've seen at OpenAI and Anthropic. You must live and breathe incidents like this. You must be really into the weeds and have a sort of an insider view of what's going on and what's happening. Can you share your views, I guess, on incidents like this, maybe a specific one, and what it means for the broader industry? [00:16:11] S1: Yeah. Yeah. Mean, if you want to go?
00:16:12 · Jack Miller
[00:16:12] S2: So as I I think, you know, anybody that reads, the the report on on the OpenAI incident, is... I don't think you can read that report and not update your views on the world. It's just such an an insane thing to have happened. And and, you know, there's there's obviously the report, then WeShed internally was mandated listening for for everybody at the company, the the three Dwarkesh episodes about about this. And so, obviously, Ryan Greenblatt one was not after the report, but but covered the same topics. So that was mandated listening for everybody. Even even the salespeople, they've gotta they've gotta have a listen to it. And I think what it shows is is a couple of I think number one, it shows that we do not have a good handle on what these models can do. It is remarkable to me that they were able to, you know, do the sorts of things they were doing with this kind of secret message board with the coordination. It it is just incredible that that occurred. I mean, using the models day to day, they still make quite silly mistakes. And then seeing that this occurred internally is just... Is actually just a remarkable fact about the world that you have this kind of spikiness and capabilities that's that's hard to wrap your head around. I think the second thing is I think the cybersecurity side is really interesting because in some sense, the cybersecurity capabilities are pretty pretty good. They're they're pretty remarkable that they were able to do this, but I think it's actually more of a reflection on the incentives that exist in the industry today. And those incentives are towards, getting the best model out that you can and not necessarily observing what's going on within, your organization. So, you know, if you read the report, you can see that actually they use AI models to discover what has happened. Right? So, in in a post post hoc analysis, they use AI models to basically go through the logs, look at the traces, determine what's happened. Now they could have easily been doing that during the time. Right? If you had a monitor looking at the way that these models were doing the evaluation, you could have seen that that this was occurring. Right? And so that tells me that it's not necessarily a technology problem. It's... It is a technology problem in some sense, but it's also a, incentive problem.
00:18:35 · Jack Miller
And so one of the big pushes that we have at HelmGuard is to change the incentive landscape for security and compliance in general, to move to a world where if you are verifying something about a company, you don't expect that to be a sample. You expect that to be on the population level. You expect everything to be monitored because it's so cheap to do so. And some of our, work with Callosum actually is bringing down the cost of this sort of agent monitoring by, like, orders of magnitude. And so to me, really, it's an incentive problem, and we've kind of built HelmGuard to change the incentives in this security and compliance industry to move towards a world where companies need to constantly monitor what's going on internally, and that that will be the de facto standard moving into the future. And I think, you know, I I think that that really is is a it is that incentive piece which is which is remarkable to me that that you can spend this much on compute. You can be running this eval, and nobody is checking what's going on. That to me is the central fact that we should be focusing on, and that we need to change the way in which the industry works from a security and compliance perspective. Because, look, this is gonna be happening in OpenAI. Obviously, they trained a persistent model. It's pretty clear that, like, there were some pretty, like, specific circumstances there. But I think what you're gonna see in, like, a year's time, these sorts of incidents, I wouldn't be surprised, occurring in the enterprise, especially with the use of open source models, especially with continual learning becoming more and more of a thing. I don't think it's unlikely that we'll see a case of this outside of the labs in the next year. And the question is, like, if you're a 10 person IT team, what... How are you gonna how are you gonna deal with that? That... It just seems like a like a very difficult problem to solve, and and I think that goes back to our thesis of bringing these frontier capabilities to the everyday enterprise. Right? Because I think they will be struggling with the same governance concerns that labs do, in in a year, two years' time.
00:20:48 · Sebastian Johnson
[00:20:48] S0: I mean, it's crazy. I mean, it's a... It was absolutely absurd what happened. How did it change? How how... Did did it change your... Both of your perceptions about the business that you're building or the scale of the problem that you're facing?
00:21:03 · John Daley
[00:21:03] S1: I I think it made concrete some things that we've held for a while, but I think have been hard for our customers to necessarily understand, and that's a lot of what we're doing in our sales process. But I think there is that, like, if you look at the traditional set of security and GRC work, one thing we're already doing there and we're doing for the the better part of a year now is saying things that were not sort of impossible, but were not economical to do are now economical because the agents can execute that work at expert level at very low marginal cost. And so that changes where you should be spending time and effort as an organization, the types of decisions that you can form all with that work. That's the kind of internal risk and security and the third party prongs of our platform. On the agent assurance and governance side, it's sort of the same idea, but different that you can only do this work if you're operating on... In, like, AI native terms. Right? You can't be keeping up with what agents are doing in your enterprise if you're not operating in an agentic way, right, on those terms at that speed with that same type of dynamic reasoning and and assessment capability. And so that is, like, concretely what it means to be bringing these capabilities to bear in a defensive way. And I think if we do that correctly, it's actually quite a... An optimistic sort because unlike a threat actors who are operating from the other side, you should be able to know everything that's going on within your organization and have that defensive edge. Right? But it is dependent upon actually bringing to bear and effectively deploying systems like ours, we hope it and others that can help you do that as, you know, what agents are running, what are they doing, and on a dynamic basis because they're acting dynamically. Is it consistent with what we're expecting and looking at at the reasoning and the traces and the tool calls. And and so both in a static configuration and at runtime. And that's really how we're extending our platform, and that's a big set of product investments that we're making for this round.