Tech Talks Daily
00:15:49 · Frank Emery
[00:15:49] S1: I mean, the easy answer is as early as possible. Yeah. But I would say it's easier said than done. What we've seen is everything from people doing security checks as you write. So validating as you type out code that what you've typed is correct, to even people integrating security directly into prompts. So for example, if you're using say Claude, having security integrated directly within the Claude that says, hey, generate a secure prompt instead of an insecure prompt. So you can start very early in the process. And my recommendation is always use sort of a Swiss cheese approach. So, you know, if you say make it so that you're doing a rigorous scan at code typing level, then you're never gonna get work done. It's too time consuming. But there's easy stuff and fast stuff that you can do at code generation time. So do it there. And then as the code is committed, then bring in scanning. And as that code moves into applications, then look at application testing, and it's becomes an exercise of understanding what is the appropriate level of fidelity and time investment in scanning for this stage of your development process. And so we look at the SDLC, from everything from cogeneration to your applications that are live in production. And there are things you can do at each individual stage that catch individual things, and have a cost, both in terms of time and complexity, appropriate for that stage. And so our recommendation is always do what's appropriate at that individual stage, at that stage in order to ensure you're being secured. To be very specific, like I said, with the earlier example, at code generation, we use something called DevAssist, that's something that Checkmarx offers that validates your code is being generated correctly. And then once that code goes into PR, we typically see people do static code, or supply chain scanning to make sure that the code didn't create any vulnerabilities, or introduce any risk within your organization. And then once that code is pushed through a pull request, we typically see people do more application testing. So whether it's an AI pen testing or dynamic application testing, that's more time consuming, but we'll catch more stuff. And then once it's deployed and fully in a runtime environment, runtime testing to make sure that while it's out in the wild, it's being used securely.
00:18:03 · Frank Emery, Neil C. Hughes
So each of those is sort of a purpose built tool for each of the steps of development. And when it's integrated correctly, it all works seamlessly and quickly. And people are usually pretty ignorant to it running in the background, unless you're introducing vulnerabilities, in which case, you'll be told pretty quickly that that what you're about to push has problems. [00:18:24] S0: Yeah. And there's also a pressure to continuously evolve and adapt as you go through any process now. And validation frameworks and security standards, how should they be changing and evolving when generated code can be produced at scale that no human team has got a chance of inspecting line by line. Any tips and advice on that side of things?
00:18:45 · Frank Emery
[00:18:45] S1: Yeah. I think the big thing is that it has to switch to more of a continuous validation framework. In the past, it was much more of a, I'd say like a stage gate for lack of a better word. And that just doesn't keep up. And so with a continuous approach, you can basically work as quickly as automation can work. And automation is going to work as quickly as AI generated code can. So I think that's the first step is that continuous approach. However, it's still important to have those checks and balances, but having them at an appropriate step. And so for example, we talked to a lot of folks who are breaking their validation frameworks into sort of a two step process. So like step one is continuous validation as code is being generated. Step two is on a daily or weekly or monthly basis, having more of an AI driven analysis to seed anything like escape, or not get caught by the continuous scanning. And then once you have those sort of two views, taking a very exploitability based approach to prioritization, which we see a lot more in the market nowadays of don't just look at vulnerabilities, but look at exploits, and what vulnerabilities are contributing to those to help you prioritize. And if you adopt those three practices, and many of those are new, then it does kind of help you scale up beyond the human limitations that exist today.
00:20:09 · Neil C. Hughes
[00:20:09] S0: And, of course, so much has changed and yet so much stays the same. And the old IT mantra of you can only improve what you measure, that's never been more important than it is right now. But knowing what to measure, that is often a stumbling point too. So what measures are you seeing that show AI assisted development is actually improving software quality, reducing real risk rather than just simply increasing the volume?
00:20:32 · Frank Emery
[00:20:32] S1: It's a good question. So AI assisted development is genuinely improving. Security outcomes when teams are moving faster. It is, like I said, helping quite a bit. I think the big thing is we do see vulnerability exposure windows are shrinking. However, we're also seeing remediation effectiveness improving. And so when we look at specific measures, I think it comes down to time to discovery is a big one. I think the volume of what's discovered, so we talk a lot about fidelity, or f1 score is another thing that we measure quite a bit. And then I would say time to remediate is another big one. So those those three areas, then AI assisted development is improving things across the board. It's interesting, like if we look a year, year and a half ago, what products existed in the market sort of before AI entered security, versus now we're seeing like marked improvements. Fidelity is through the roof better, AI is helping us find more and faster. We're seeing remediation windows or remediation periods now, drastically improving alongside the exploitability windows, is fantastic. So things are being exploited faster, but they're also being fixed a lot faster. So And those are some of the measures that that we look at, and they are all genuinely improving, which is good to see.
00:21:47 · Neil C. Hughes, Frank Emery
[00:21:47] S0: It really is. And for people listening, if any of our conversation resonated with you today, the good news is you're not alone on this journey. Checkmarx, for example, helping the world's largest enterprises get ahead of the application risk that we're talking about without slowing down development. So for anyone listening that would like to carry on that conversation, where can they find out more about everything we've talked about today? [00:22:10] S1: Yeah. The best place is our website, checkmarks.com. We have a lot of information about AgenTek application security testing, as well as events we're going to, webinars we hold, lots of goodness there. [00:22:23] S0: Awesome. So much great things in what you've raised today and great work as well. I really support what you're doing here. And, Frank, I will include links to everything everyone's mentioned... That you've mentioned. And for people listening, please go check it out. Continue the conversation. But thanks for bringing it all to life today, Frank, in a language everyone can understand and regardless of age too. Hopefully, people find value in this. But thanks again. [00:22:45] S1: Absolutely. I'm happy to be here.