Tech Talks Daily
00:09:01 · Frank Emery
[00:09:01] S1: Ultimately, it's the organization. So at the end of the day, if you're the end user of a product and that product gets breached or hats, you know, you're not necessarily gonna know the individual team that dropped the ball, but you are gonna know the organization that dropped the ball. And so I think ultimately, you know, the buck stops with the the company that's putting products out. I think within an organization, if you look at the individual team levels, it's gonna come down to, if you're having an AI write your code, you know, you're responsible for that AI's writing of your code, and that all bubbles up. And so the individual developer, they're accountable for validating that whatever they wrote is secure. And that's, you know, to the original point about why that skill set of validation review is so important now. And then if you're a team leader, you're responsible of making sure that every developer on your team is validating their changes. And if you're running an AppSec organization or an AppSec program, your response for making sure that all those team leads are adhering to application security standards. And so it does all ultimately bubble up, but that is still a lot of points of failure along the way where as people speed up and as they lose the ability over time to do, you know, reviews, security architecture, all of that just increases risk across time. And unfortunately, AI is making it a lot easier to exploit things, but at the same time, it is helping both with detection remediation as well. So it's not all bad. There there are ways to fix this. And we're seeing a lot of organizations starting to actually use AI to help with detection remediation to keep up pace with vulnerability creation as well.
00:10:39 · Neil C. Hughes
[00:10:39] S0: And if we do have a young Gen Z developer who's entering the workplace for the first time, maybe even going to an interview and wanting to make the employer understand that they know what they're talking about and what they're responsible for, what should they be expected to understand about reasoning, validation, secure design when they're orchestrating AI generated output rather than writing every single line themselves like it used to be done?
00:11:02 · Frank Emery
[00:11:02] S1: I think first and foremost, they need to understand what the code being generated does. I have seen far too many instances of people firing off a prompt and getting something back and immediately executing a pull request. So they're they're going right from prompts to deployment. And when you do that, you're not necessarily understanding what that code is doing. And a lot of the times, vulnerabilities aren't necessarily just across a scripting vulnerability, they're more logical, they have more to do with what that code does in context. And so I think if you're a new grad, if you're new in the industry, I would say don't just trust what AI is doing blindly, you have to understand what that code is meant to do, how it fits into the overall design of your application. And I think you need to have a good idea of, you know, what secure design looks like. We see a lot of blogs online that'll talk about, hey, AI passed this functional test. The AI made code and it did what it's supposed to do. Like if you press this button, the AI made that button will do what you expected the button to do, but there's a big difference between being functionally correct and being secure. And I think that's where people really need to do research and figure out and understand what that that difference is and and how to make the the code that's being developed by AI, not just functionally correct, but secure as well.
00:12:30 · Neil C. Hughes
[00:12:30] S0: And that is the young developer lab. An organizational level, how can they decide which changes an AI coding tool should make independently while others will still require that human review? It's a
00:12:45 · Frank Emery
[00:12:45] S1: good question. We actually talk to a lot of customers about this. And I would say that what you let AI do independently will depend a lot on how your organization has implemented AI across the board, how mature you are. I've talked to some organizations that have entire teams that are just AI. Like there's there's one company I've talked to, they have a team where it's an AI, that defines its own requirements, writes its own code, implements that own code and push into production. And that's one end of the extremely mature spectrum. And then you have the other end, where organisations aren't even scanning their code for security risks, AI generated or not. And so I think when you look at, you know, what should AI be allowed to do independently or not, you have to start with what control plans exist within your organisation, so that when AI takes actions, there's something in place to make sure the actions it's taking are correct and secure. And so this could be anything from, you know, if you have automated scanning and automated remediation in place, then great, you can probably let AI do a lot more than it would in other situations. If you don't have any of that in place, or if your scanning remediation really aren't up to where they need to be, then then you probably have to have a human be more involved. And so I think that's sort of one way to look at it. The other way to look at it is sort of the idea of routine versus more advanced work. I don't think AI is at the point yet where I would let it go in and define architecture for my business overall. I think it can certainly help in the process. I But think that still needs human review, but low risk changes. I'm even seeing people in product do this now where I remember back in the good old days, if you got something built, and you looked at the demo, and you saw the colors were wrong, you'd have to cut a ticket, and it would take a week to change colors. Like that is low risk. That's easy. Automate that or have non technical audiences do that. It's pretty easy. The architecture side of it is is much more risky, is much more critical to a business. And that's where I'd still have humans be in the loop. And then the other part that's kind of interesting is around testing. You know, we do see AI coming up more in security testing.
00:15:04 · Frank Emery, Neil C. Hughes
But we see a lot of organizations doing is adopting a policy of separation of Trish and State, where the AI that's doing the development can't be the AI that's also doing the security testing because they'll cheat. So, you know, using AI as part of the security testing process as well, but making sure that it's a different AI process. That's part of that so that you can ensure that, you know, you're mitigating risk. [00:15:30] S0: And a time where many leaders are feeling the pressure to move forward more quickly, security controls can be seen as something that become a a late stage gate that's actually slows down delivery. So where should checks sit so teams could reduce application risk without losing that speed that AI provides?