Karissa Breen [00:00:10]:
Welcome to KB on the Go and today I’m coming to you with updates from J. Frog’s Every Ops Day Conference. I’m on the ground at the Swiss Hotel here in Sydney. Every Ops Day is a premier one day conference dedicated to exploring the latest strategies, technologies and best practices in DevOps, DevSecOps, AI and MLOps. This event brings together industry experts, thought leaders and technology professionals to discuss how organizations can optimize their software supply chain security and innovation. So for now you’ll be hearing from a few executives on their thoughts towards managing Every OPS and key insights around Mida and a cloud migration journey. KPI Media is bringing you all of the highlights. Joining me now in person is Sunny Rao, SVP Asia Pacific at JFrog.

Karissa Breen [00:01:03]:
And today we’re discussing how ANZ tech leaders are adopting EveryOps and how to strengthen software supply chain security. So Sunny, thanks for joining and welcome.

Sunny Rao [00:01:12]:
Thank you. Pleasure. Pleasure to be here.

Karissa Breen [00:01:14]:
Okay, so Sunny, obviously it’s the end of the day. We’ve had a big event today, a lot of people talking. So maybe let’s start with your view on what’s broken in the software supply chain right now. And maybe if you want to focus it across the Australia and New Zealand.

Sunny Rao [00:01:27]:
Market, I probably wouldn’t use the word broken. We are always trying to strive to become better as organizations in general. So it is an evolution of practices. If you look at how application development or code generation and the process of building and distributing code has evolved, we’ve also evolved from the age where we wrote it on mainframes, to PCs, to the web, to smartphones and now into Genai. So the, the technology underlying that has changed quite a bit and therefore how you write software and how you distribute, that is changing. And so it’s an evolution. A few years ago, application developers or developers wrote code, then they had to wait for resources because you needed infrastructure to be able to run it and someone had to size it. And there was a team that was specializing in sizing the infrastructure required because most of it was inside the organization.

Sunny Rao [00:02:22]:
But then came the whole concept of the cloud and the migration of workloads into the cloud and it almost felt that the resources were available on the tap, so to speak, right? So you can get into it and you can write your code and then suddenly all the infrastructure needed is available in the cloud. So it almost felt like you really didn’t have to go and ask someone for resources. It just happened. What that meant for organizations is the developers were able to do more than just write applications. So you went from application development to release, you went into monitoring as well. So the app developers started to take on more of the workload. But then of course you had security as a completely different function inside an organization. And so developers would build and then at some point there would be a gate where security would check it.

Sunny Rao [00:03:17]:
The amount of open source code started to increase, so your first party code became smaller and smaller. Third party code coming in from open source repositories became larger and larger to the extent that nearly 80 to 85% today, if the industry releases its data, it’s between 80 to 85% of anything that developers are writing is open source. Now obviously with open source you are bringing in a lot of packages into your organization. You continue to build and then you hit a security gate and at that point things start to fail for whatever reason, licensing policies, vulnerabilities, et cetera. And then you have this immense amount of rework that needs to happen. But the problem is by the time the security team is actually going through its analysis, the developers have moved on into the next sprint. And now suddenly the next sprint starts to get delayed and they have to come and do rework to fix all the vulnerabilities that the security team reported. So I don’t know if the workflow is the most effective in that manner and it’s not the best way to work.

Sunny Rao [00:04:19]:
So modern day practices start to give more alerts to the developer at the time and have policies for organization that would allow you to determine what needs to come into the organization in the first place. So if there are certain licenses that you do not want to have included in your code, then why get it in the first place? And that’s kind of where we start talking about better work practices, better DevSecOps practices. So you can curate what comes into your organization through your policy management. So once that is done, as the developer starts building, wouldn’t it be great if we could tell them what vulnerabilities exist in the code that they’re writing, especially in their dependencies. And then you have transitive dependencies that are in there. So if you could find flag it to them, of course, with the remediation necessary as well, then when you actually get to that security milestone, most of the times it would pass the gate. And so the way you have seamless delivery and deployment of software would be much faster. And that’s really what it’s all about.

Sunny Rao [00:05:23]:
I mean, I think everyone calls it the cloud operating model at the end of the day.

Karissa Breen [00:05:27]:
Yeah, so it’s Interesting that we’ve seen like. So you spoke more about like that waterfall approach. Like we, you’d run a whole project, get to them, you have to pen test it and look everything from a security perspective. Then we started to integrate SEC DevOps, then we’ve got security champions. Where do you think in terms of operationally we can even increase that even more? Because now with everything, you know, we have to push things faster. People want new features and functions to go out quicker than before. What are you sort of seeing given your role in your region that companies are starting to adopt?

Sunny Rao [00:05:56]:
Yeah, I would say the tool chains are getting much better. So when you start to take the tools sprawl out of your organization and start to get better traceability and visibility and integration of all your tool sets, then it starts to become much easier. Look, you can do a lot of it today, but if you have a lot of tools in your organization, then you have contestability. How do you determine what is right? If tool A says you have 10 security vulnerabilities, tool B says you have 20, which one’s right? And so you get into this recourse of figuring out what do I trust, what do I believe? Now, at the end of the day when you write applications, you start with source code and then you get into binary management. All binaries are stored at the end of the day in the JFrog platform, in the artifactory. So when you’re starting to look at binary as the key source or system of record or source of truth for your organization, then when you put in all of the security gates as to what vulnerabilities exist in your packages and its dependencies and then you make it contextual. Not everything is a vulnerability in your application if you’re not calling certain dependencies at all in your application. So today what happens is developers go and get some packages, they put it in, they write the code, they submit it, you may get a whole bunch of vulnerabilities, but many of those are not even relevant to your application because you’re not calling that class file or you’re not calling a certain library within your package.

Sunny Rao [00:07:28]:
So why do we stop work to go and fix that if it is not even a problem?

Karissa Breen [00:07:33]:
So one of the presenters today I’ve actually interviewed on the podcast, we spoke a lot about S boms. So how do you think that sort of fits in with what you’re discussing today around if things aren’t relevant, like do you really need to look through under the Microsoft to see like what was there? Or how do you sort of then contextualize it to your point on, hey, there is some vulnerabilities, but it’s okay, maybe there’s not a huge risk. How do you get to that stage in terms of mindset?

Sunny Rao [00:08:00]:
Yeah, and that’s where contextuality comes in. But also further gates come in. We talked about curation. How do you check before things come in great, you’ve got passed through that gate. How do you check as you build, how do you check that it is relevant to what you’re building? So applicability, contextualization, and let’s say at that stage, some of those risks are not relevant to what you build. Now you’ve gone on to production, so you pushed it into the runtime. Now when you’re in runtime tomorrow, there may be a new vulnerability that crops up and that’s what happens. But that is now relevant to what your application is or what your code base is.

Sunny Rao [00:08:38]:
So now you need to trace it back and to say, okay, where did that problem emerge? Who was that developer, which team was it? And what do we need to fix in order that we don’t have to shut down a runtime or a production instance in order to remediate it? So how do we get that through quicker and get that update quicker? So you need to have this end to end view and it needs to be constant and you need to have traceability. And SBOMs are basically at the end of the day, you’re getting something into production. You’re basically stating what’s in my packages and what’s in my software, what kind of license I have. Some of those are compliance requirements, but then there are security requirements as well. And that’s where traceability and evidence management comes in end to end from ingestion all the way to runtime.

Karissa Breen [00:09:25]:
So going back to your point before around maybe it’s not an issue today, but tomorrow it is. Is that when the alerts come in to say, hey, there’s an issue or else, I mean, if you had to manually track it back and then, you know, the person doesn’t even work anymore, you can’t ask them is that what. That’s then removing a lot of that manual traceability and people not having full fidelity of the story, who did what, et cetera.

Sunny Rao [00:09:48]:
Absolutely. You’re spot on. Yeah, spot on.

Karissa Breen [00:09:51]:
So with that in mind and the way in which the, you know, the market’s moving with the lower barrier to entry in terms of people developing code, people pulling code off, you know, open source repos, how do you see all of this fitting into day to day operations for developers now.

Sunny Rao [00:10:09]:
So ideally developers want the least amount of friction. They need to move fast. Most developers will tell you that it’s not the tool set that slows them down, it’s the workflow that slows you down, which means that you’re handing off to another team to do some other checks and they come back well after the sprint is over in asking you to remediate. And that’s a big deterrent because product managers who are managing this have a deadline to meet about a certain release and we slow them down. So where it is changing is to give visibility to the developers from source code through to binary, so that they can actually remediate as they build.

Karissa Breen [00:10:49]:
So do you think with that, I mean, okay, so if I zoom out, I go back to comp Sci degree. Traditionally they don’t teach you anything about security. And I’ve worked with develop, I’ve been in these roles before, I’ve worked directly with developers trying to educate them at cybersecurity, the security champion and all the stuff that we’ve seen to get them to this point, historically speaking. But do you think it’s just not in their repertoire to think like a security person? Because a developer by trade. So how do you get it to the point where it’s like, it’s not so binary? It’s not like I’m security, I’m a developer. How do you get it where it’s like, okay, we’re working in harmony and as a unit rather than the silos. And I know it’s changing, but I’m curious to hear your thought.

Sunny Rao [00:11:27]:
It’s not about so much the security mindset, but more about the accountability mindset. So if I’m writing code, I’m accountable for the quality of the code. The quality of the code just doesn’t mean how functional it is. It’s also how secure it is, how reliable it is. And that’s really what these tool sets are allowing you to do, is giving a lot more of the empowerment back to the developer because they understand the context of what they’re developing. When it gets to security, it’s very binary. Here’s a problem, go fix it. Then the developer comes in and says, but you don’t understand the context of that.

Sunny Rao [00:12:01]:
That that problem that you’re reporting is not even relevant to what I have built. It doesn’t even come into what I’ve done. Why do I need to go and stop all the other workflows in order to fix something that is not even relevant. And so contextuality becomes much, much more relevant. Who understands context better than the developer who’s building it?

Karissa Breen [00:12:20]:
So the interesting, and probably a common undertone across all the interviews I’ve conducted today would be, like we said, you’ve got the project manager, you’ve got the developers, you’ve got the security people, the system owner, all these different people in the equation. Right, but how do you get it where everyone’s on the same page? Yes, security is important. Then we’ve had issues in the past where it’s like security, like police people, then developers like, well, I’m getting whipped by my project manager, that’s KPI’d. And if they don’t hit the KPI, then they’re on a contract, their contract’s done, so they have an agenda. Then the general manager’s got an agenda and he’s not going to get his bonus because we’ve got to push all these things live. So how do you get it to a point? And these are real examples that I’ve actually sat in. And then it starts to feel like, what is the main mission that we’re here to do? It’s kind of like everyone loses sight of that security people going, well, I’m here to do security. But then they don’t really understand the developer’s mindset, who understands the business and the requirements.

Karissa Breen [00:13:10]:
How do we just get on the same page and operate as a team rather than everyone’s got their own agendas. And then as a result, we don’t get anything done.

Sunny Rao [00:13:19]:
And a lot of those discussions are getting elevated. You know, when it gets to the CIO and into the board, they start to get elevated. What are your goals? You know, what is my time to release? What is my vulnerability? You know, the security posture inside my organization? What is my risk score? These are the questions that are being asked inside, inside the senior leadership of the organization. And so people then start to look down and say, how do I make work happen in a more effective and efficient manner? Now there will be a phase of time, and whether it is code development and security or any other organization, there are organizational challenges that are there. These are people challenges. But that’s where leadership comes in to identify and say, these are our goals. Here’s what’s slowing us down, here’s how we’re going to change the way things work. And that’s where we are starting to see an increased adoption, even including in particularly also in Australia and New Zealand, the increased adoption of a single source of truth, the Ability to firewall what’s coming in in order to make lives better for developers, greater empowerment of the developer in order to build and take more accountability of it and reduce tool sprawl.

Sunny Rao [00:14:31]:
Because then you can actually make people more empowered. When you have 10 different tools, who is the owner of those 10 different tools?

Karissa Breen [00:14:38]:
That’s a good point. So going back to just for a moment, workflow. So it was coming my mind as you were speaking there Sunny would be, I don’t know, you’re building a house and then all of a sudden there’s an issue. Then you got, you know, some, some oh and s person has to come in, triage it, look at it, call some other person, that’s another part of the world to get their view. Then I got to write a report, then it rains. All of a sudden your whole house project is delayed like multiple months because of whatever’s happening. So I’m guessing that the same sort of mindset and approach is injected into developers, like you were sort of saying. So then would you say that that creates a lot of friction then with the development community, then with the other parts of the business?

Sunny Rao [00:15:18]:
You would eliminate a lot of friction if you gave everyone end to end visibility. Because once you have visibility and you have the single source of truth across it, then it becomes easier for people to understand. So you know, you talked about the conflict between the developers and the security personnel, for example. But if everyone understood that this was not a security risk, despite the CVSS score being high, because in the context of the on the code, I don’t believe that there are people in the inside security team who would go and say no, it needs to get fixed irrespective of that, if it’s not even relevant to what is being rolled out. Because they too have the best interest of the organization, which is you do want to get something released, you need to get something out into production. And people are not going to stop work for the sake of stopping it as long as they have visibility and insight. The problem you have today is if you don’t give that visibility and insight to the different parts of the organization, then you have contestability.

Karissa Breen [00:16:15]:
Why would you say historically this hasn’t happened?

Sunny Rao [00:16:17]:
Organizations have grown, there are lots of tools. These tools are not integrated to each other. Each one has been taken in a silo. And you’ve almost tried to take a waterfall approach and then go into an agile work practice. However, the tool sets were built for the waterfall method, not for the agile mode of development. Inherently by doing that, you are starting to look at small pieces. So for example, in an organization, you may have one tool that just does secret detection, you have another tool that does sast, you have a third tool that looks at source code vulnerability. So you have all these tools and the security personnel themselves are overwhelmed by, by, oh my God, I’ve got a vulnerability in source code.

Sunny Rao [00:17:03]:
I have a secret embedded somewhere. I’ve got SaaS telling me there’s something wrong. Where do they go? Who do they go to? At the end of the day they say, well, we can’t, we got to scrap this, send it back to the developers, get it fixed. But who in the developer? So it’s almost like the whole project has stopped and that creates a lot of friction. But when you actually give people visibility to where the problem lies, and if you can remediate more of that work as you’re building it, then it makes it much better. And to your point of the analogy that you use of building a house, the whole reason why we have standards in building codes is for that reason, right? So you take away rework, you take away issues of compliance later down the road because it’s far too expensive to fix it later than to fix it when you’re building it.

Karissa Breen [00:17:47]:
Yeah. Okay, this is really interesting. So I want to, what I’m curious then to understand in terms of the traditional side of things. And I know it’s because I’ve worked in these roles. So for me it’s like there’s this seeing that change. Where do you think we’re going to get to now in the future? Because as we know, you know, we have to push things faster with more competitive landscape out there now, even for businesses that we’re supporting. Where do you think this is going to go? And is it, is it going to become like there’s always going to be some level of demarcation between a security team and a development team. And yes, that’s becoming not so that much of a delineation between the two.

Karissa Breen [00:18:23]:
But where do you think it’ll get to moving forward? Where people are on the same page, they do understand, irrespective of the developer security person, how do you see that moving forward?

Sunny Rao [00:18:32]:
There’s a lot more than just the developer and the security. We are living in a world where more and more of what a business does is composed of software. It doesn’t matter whether you’re in mining or whether you’re in technology. The amount of software that an enterprise builds is a significantly larger component of its business. In many cases, it is the business so if that is the case and you’re running in public clouds, your own private clouds, but at the end of the day it touches the consumer. You also have regulation, you have governance, you have compliance. So it’s a lot more than just security. And especially if you’re now starting to use machine learning and AI, there’s privacy.

Sunny Rao [00:19:10]:
So you now have a lot more layers coming into the picture. So I would say that just worrying about developer and security, I think it’s a little bit behind us that needs to get solved, if it hasn’t already been solved with, with the right platforms, but then very soon you are now having to comply with governance. When I write my software and I release this in Europe, what am I complying to? When I release it in North America, what am I complying? How do I provide you evidence that I have actually complied to it?

Karissa Breen [00:19:40]:
So I think my question was more like how? Because we’re going to have all these new components coming in, what can we sort of learn historically perhaps from the friction between devs and security folks to not make that mistake again?

Sunny Rao [00:19:52]:
And that’s where we go about in saying have a single source of truth. When the reason for the source of truth is to provide visibility, friction happens because people are not on the same page, they don’t see the same metrics, don’t read it the same way. So if you can actually give that visibility and insight through one common layer of understanding, that allows you to then drill down into where the problem lies, which is traceability. When if you can say that I have followed all the policies of the organization, which is evidence management and audit, then it is no longer an emotional discussion, but a more objective identification of what needs to get fixed. And it also helps you build greater automation because everything is very objective, there’s no emotion in it. And therefore what you really need is a platform that a democratizes work through integration with all the tool sets that you use, provides one source of truth, one system of record visibility, end to end, irrespective of whether things are in your platform or coming from in outside or going out with the metadata the right way. So giving that end to end visibility, when you have that, it’s very easy to solve problems because everything is data driven, it’s objective.

Karissa Breen [00:21:08]:
So going back to the visibility part, I mean over the years you probably would have heard like, oh, you know, this person’s talking this language, another person talking that, but we’re not understanding each other. So do you think everything you’re just saying based on where the industry’s going like that won’t be a thing anymore. It won’t be. You’re talking this language because it’s going to have the visibility, the traceability. Therefore everyone’s seeing from the same hymn book regardless, because everyone can see it. So there’s no, there’s different languages. And all that stuff we’ve spoken about for the last 10, 15 years, that’s going to be obsolete.

Sunny Rao [00:21:34]:
That depends on the adoption. So it’s not a utopian answer that, you know, just because I have this platform. One of the reasons why JFrog went out and did the whole integration with GitHub was really to provide the developers that view from ingress all the way down to binary, so that when they’re building their application through open source and they’re actually adding their own code, they can actually see whether there are issues not just in source code, because now you’re in a GitHub environment, but also into the binary. And to do that with a wider view, because JFrog is showing you not just whether it’s contextual to your application, but, but whether this is even relevant to your industry. And I think that’s one of the things we highlighted today. If you’re building something in financial services, are there other financial services in organizations that have used all these packages, have they found issues for you? And that’s why we put so much investment into the research team that’s constantly looking at public repositories and what vulnerabilities exist and which industries are affected, not just for your open source, but also for open source models.

Karissa Breen [00:22:40]:
And that’s where the contextualizing piece comes into it.

Sunny Rao [00:22:42]:
Absolutely.

Karissa Breen [00:22:43]:
So in terms of the boardroom conversation, what does that look like? Why now has this been elevated to like a board level conversation?

Sunny Rao [00:22:54]:
Because everyone’s looking at how much does it cost for me to roll this out. It’s always a. It’s a money problem and money problem, because why does it take me so many people to produce this output? Why do I need more people? I don’t seem to be making progress. I seem to be doing constant engineering, but I don’t seem to be doing any releases. And when you go back to our conversation that’s linked to how many times I’m just reworking on something that I already released and put the new Sprint on hold, and then engineering teams come in and say, I need more people. Every release that goes out there actually compounds that problem if you don’t have these effective tool chains.

Karissa Breen [00:23:33]:
So I just want to stay with you there for A moment. What was coming in my mind was the contextualizing part. I’ll give you an example again, I’m not a doctor. Doctor, I’m sick. They’ll say like some scientific word, it’s like, yeah, but what does that actually mean then? Let’s say you have to do this whole thing in this program for the next six years of your life, or depending on how sick you are, it’s like, I’m not a doctor, I don’t know. So do you think sometimes boards overlook some of these things because how could they know, right? They’re not a developer, they’re not a tech person. Like, they don’t know, like the intricacies of what goes into building something which may look simple or a button on the screen to give you their proverbial example. But do you think there’s a lot of that in terms of it just is easily to be overlooked? Oh, why is it, why can’t we get this faster? Why is it there so many people.

Sunny Rao [00:24:17]:
Boards ask money problems. Boards don’t ask about why am I rewriting the code three times before release. That’s a time to release question. That’s one of the byproduct order or metrics that they do measure, but that’s a proxy to how much does it cost me to roll something out. So you know, all of those questions about rework or time to release, my security posture, why am I paying millions of dollars on 20 different security products? What’s going on? But I’m still, you know, I’m taking down applications, I still have breaches, I still have AppSec violations. What’s going on when I’m putting so many millions of dollars into all these different tool chains. So those are the money questions that are coming in. Your expenditure, your engineering cost, your infrastructure cost all seem to be going increasing.

Sunny Rao [00:25:00]:
So what’s going on and why.

Karissa Breen [00:25:02]:
And so then what are people, what’s the response to that?

Sunny Rao [00:25:04]:
And that’s where people want to go and tell you, you know, what’s my time to release? Why am I, why do I need 6000 developers with my time to release continues to go up?

Karissa Breen [00:25:12]:
But do you also think there’s this, there’s just a gap in knowledge there?

Sunny Rao [00:25:15]:
I don’t believe today that it’s about a gap in knowledge. It’s a gap in getting the organization line to actually go and solve. What is blocking you from becoming more effective? I don’t, in my experience in working with enterprises here, especially, especially here in Australia, I have not seen that it’s more inertia about making the change. It’s a change management issue rather than lack of knowledge. Everyone knows and the market’s getting educated. Their peers are talking about it in the industry. They’re telling you where the gaps are.

Karissa Breen [00:25:46]:
So in terms of final thoughts, closing comments what do you think’s on the horizon next?

Sunny Rao [00:25:53]:
We are at that stage where boards and I’ll talk to you, boards are going, well, I’ve got this new wave of gen AI. What does it mean to me as an organization? I hear things that we should be able to go 30% faster, 50% faster with 50% less resources. Is that true? Is that going to happen? But all of those questions will only accelerate. Focus on having much more efficient development, distribution, monitoring processes inside organizations. Because the pressure is going to come. We need to move faster, we need to move at lower costs. We need to be more effective, do more with less. Those questions are going to keep coming and so the pressure will come for organizations that are resistant to change to go out and make the change.

Karissa Breen [00:26:51]:
Joining now in person is Craig Wilson, principal Cloud Platform engineer at iris. And today we’re discussing how IRIS modernized its infrastructure through cloud migration to overcome legacy constraints. So, Craig, thanks for joining and welcome.

Craig Wilson [00:27:03]:
Nice. Thank you.

Karissa Breen [00:27:03]:
Okay, so I’m curious to understand from yourself, Craig, talk me through existing infrastructure and what was sort of the bottleneck. Now, it depends on who you’re speaking to. Some people say technical debt, slow release cycles, or just even the weight of like regulatory sort of pressure. Right. So maybe let’s zoom out, talk about general cloud migration and then we can maybe hone in on the JFROG part.

Craig Wilson [00:27:25]:
Yeah, look, definitely we like Iris. Probably more than 10 years ago, probably even further than that, we started a cloud migration project where we recognized that, you know, we had, we have data centers, physical data centers with physical machines. But then when you’re onboarding a new client and you don’t have the hardware there, you know, it would take months in order to procure the hardware, get the highway in, get it set up, get it, you know, get it all ready to go, get the software in, and then the client can actually start using the software, not to mention actually configuring that software as well. And that added to the lead time. So it was recognized that, you know, can we, can we cut all that out and just have the ability to actually deploy the software and get it configured faster and get it in the hands of our clients a lot faster? Additionally, all that hardware, obviously there’s a turnover that you’re Having to deal with hardware that’s failing hardware that’s, that’s ending its life cycle and out of warranty having to keep up with that. It was, you know, a big challenge and we wanted the company to move faster and focus more on the clients rather than the infrastructure. So we went on a journey to pick a cloud provider and then we started changing the way the business operated. You know, we, and that changed a lot of methodologies.

Craig Wilson [00:28:35]:
We went from software development lifecycles that were a lot longer. You sort of still coming out of that waterfall model type pattern and you start to hit more of the agile methodologies. We started to hit new technologies around containerization to help speed up a lot of the processes, have consistency, have it so that developers can run the same thing locally as you do in the runtime environment and trying to keep ahead of all the modern practices that are coming out, coming out faster and faster. The world was changing at that point and we started, you know, we’re in the cloud, we start running things with cloud. So you start getting confident. Well, I can do lots of other things in the cloud, like we’ve got other systems that we’re running, you know, with developer tooling. So instead of running that on physical hardware, let’s run that stuff in the clouds, you know, so we start doing that, we think, cool, we can do that, we can manage it, we can look after it. But after time you realize now we’re spending a lot of time managing this infrastructure for this development tooling and that is taking time away from, you know, new technologies tool, new tools that developers want, honing in and really optimizing the software development lifecycle and dealing with security.

Craig Wilson [00:29:37]:
That’s coming up more and more as developers start using third party packages. It’s become a bigger problem. You’re chasing constantly around what’s, you know, I need this new update because it’s the newer version, but I need this one because it’s a vulnerability. But do I need to take it because is my actual software vulnerable to this version or not? I know it’s saying it’s got one, but is it applicable to me? It starts to wear you down a bit by bit. So really when it came also now circling to the JFROG perspective, I had personally set it all up, the last setup, architectured it and put it in place and you know, for the most part it was, you know, it was manageable but you know, there was a lot of domain knowledge that I had learned about how that works. And you know, it becomes difficult to pass on to others when they’ve got their focuses elsewhere and it’s not their primary day job. Also again, like yeah, just managing upgrades and managing where when there’s instability issues and trying to chase up and scale, like having to deal with the fact that we’re constrained by licenses and we can’t deal with the load sometimes as much as we would like because we can’t scale out as fast because we’re constrained. So we kind of recognize that we don’t want to spend all this time looking after it to that extent.

Craig Wilson [00:30:45]:
We only have to always look after configuring it and that’s always going to be there, but we don’t have to look after the infrastructure. We had a few attempts of moving it to SaaS with JFrog. The first attempt they were just really dipping their toes into it and they weren’t quite ready. It was like a they half their services were available and the other half weren’t. So we’re like, well there’s no point. I don’t want to manage a hybrid environment so we just manage it ourselves. It’s a lot easier. The second time, budgets just didn’t align.

Craig Wilson [00:31:11]:
We just couldn’t get it across the business. But then this third time, which happened towards the end of last year, the business came on board and said yeah, let’s do this. And we signed on and because of all the features that came out of that are ingrained in the platform, with our J4 platform we were able to turn around a migration within, I think within two months we had it ready and after five months we’d shut off our self hosted environment completely and there hasn’t really been any major problems. It’s just been smooth sailing previous to that where we had moved from, you know, physical servers to into AWS or into the cloud environment and self hosting it. That took eight months this time. Yeah, it’s was pretty quick, which is really good.

Karissa Breen [00:31:54]:
So I know that you’ve been with Iris, I think like more than 20 years. So you’ve probably seen quite a substantial shift going back to, you know, the company move faster. So would you say, given your experience and a tenure with the business, that nowadays and where we’re at in 2025 we are seeing companies move faster in terms of deployments, releases, features, functions need to push quite quickly. Where do you start to see perhaps some of the, the conundrum sort of start between companies wanting to move faster but we need to cover that up with security, you know, DevSecOps et cetera and MLOps so how would you say, given your experience, Craig, can people sort of work in equilibrium with the two?

Craig Wilson [00:32:35]:
We get people wanting to do things which you’ve never heard of in the past where like, I want someone to start it today and I want them to push something or create something new or put something into production on the same day. I’m like, cool, that sounds all great and it sounds awesome. But then there’s a lot of things that come to play before you get there, like dealing with security. What have they written? Is it got any vulnerabilities? Has it passed all these tests? What are you doing with tests? What do you, has it gone through compliance? Compliance can be. Regulatory compliance could be. Have you got backups going? Have you got monitoring going? Have you got all this stuff like how do we get. Make sure all these things are in place to know that this thing. Yes.

Craig Wilson [00:33:09]:
Is definitely production ready and can you do that within a day? Within a day. It seems like a very short time frame, but we need tools to actually help speed up this process and make it a lot, a lot easier. I mean, I think, and that’s the struggle at the moment is there’s so many tools and every company, every vendor is trying to do everything they’re trying to do the whole end to end and you end up in an overlap of which one should I use, when should I use it, Have I got full coverage? How am I sure that this thing that I’ve now put in production has gone through every single one of these gates to get there? So we’ve talked about our company around attestations like how do we, how can we set up an attestation service to actually sign off along all the stages? And we know that we can put a gate at the very end saying if this thing hasn’t gone through all these gates and it hasn’t been signed off by them, then it can’t go out the door. How can we set those types of systems up? Because you need those to go faster. Because the end users, they want to go faster. They want to. Even right now they’re trying to use every AI tooling under the sun. That pops up, something new pops up.

Craig Wilson [00:34:07]:
Yes, I want to use it. But then now the next, next problem of we’ve now got to, you know, check that for compliance. Where’s the data going? You know, is this, is this something now we have to procure? Is it. We want them to move fast and we want. But we also need them to move safely. And unfortunately, sometimes that may mean that they don’t always get the tool that they want, but we’ve got to make sure that the tools that they have actually satisfy their needs to the best of the ability.

Karissa Breen [00:34:30]:
So word you said is like, tools. So what I’m often hearing in the market at the moment, you know, I’ve spoken to many sizes that are like, hey, we want to move more to, you know, platformizations. We want to move more towards observability, interoperability. We’re hearing all of these things. So how do you get to that point where you’re like, okay, all these tools in the tool shed, how do we start to make less tools? How do we get to that point? And how would you say, given, you know, what you’ve just shared with me around, like, our business wants to go faster and we need the tools to do that, but do we need all of the things? So how do you sort of, how do you look at that not just from a vendor perspective, but from an internal perspective as well? To be like, we kind of need it, but maybe we could forego it because we can’t have all of these tools not talking to one another. Right?

Craig Wilson [00:35:11]:
Yeah, I think we need to, like, you need to assess exactly what you need at every single stage. Assess what tooling you already have and whether it covers it adequately. Try and focus on the gaps you’ve got. Sometimes we, you know, you can focus too much going, oh, we need this new tool here. But we go, well, we’ve already got three versions available. This is not really a primary focus. Like, we need to focus on the ones that we’re missing that provide a better end to end experience or coverage as we go to production.

Karissa Breen [00:35:39]:
And so just, so just to jump in there, when you say, like, we’ve got three of the same thing, is it because perhaps someone’s come in from a previous company, they’re like, I’m familiar with X tools, so therefore I want it over the other three we’ve already got. Or where do you think sort of that mindset comes from?

Craig Wilson [00:35:52]:
There’s a lot of that. Someone new comes in, they’re always. There’s something they’re more familiar with and they want to use it and they push it. I want to use this, I want to use this. It’s like, we’ve already got these other ones and maybe they don’t understand that those other tools will satisfy their needs or they’re just not comfortable with them. Maybe that tool is lacking and we need to actually reassess it and change it and make that decision. But that decision doesn’t always go fast because you’ve got procurements, you’ve got contracts and you’ve got to wait for those to expire before you can, you can switch over. But also just the tooling you’ve got.

Craig Wilson [00:36:17]:
Like I’ve. I say a lot how my initial example is around secret scanning. All the vendors started getting on board with secret scanning. We’ve got secret scanning, we can scan these secrets. GitHub’s got it. We use Git Guardian. That’s got it. JFrog’s got it.

Craig Wilson [00:36:29]:
Everyone tries to do it. It’s like, but hang on, I don’t need everyone to do it. But where is the right places to do it? I mean having at the source code level, that makes sense. Having it now I’m scanning containers that have been built. That makes sense. So you gotta make sure you tick off all those boxes and make sure you’re doing it the right spots. But we don’t wanna have to deal with overlap and have to have to deal with multiple alerts going off at the same thing all the time. We need to try and get back to a single single paying last.

Craig Wilson [00:36:54]:
But it kind of has to also work across vendors. And I think sometimes we need some sort of. In some parts of the industry, you know, they’ve come up with frameworks of interoperability where they’re working together. It’s a common protocol. I’ve got my tool here, but I implement this protocol and your tool’s over there and we can work together and it can come together as a single pan of glass. I think there’s a lot of tooling that probably needs an inflection point of getting, getting there so that we can. So it doesn’t matter if I’m using tool A and you’re using tool B. We can both use tool C and it all works together as one.

Craig Wilson [00:37:27]:
Which is what I sometimes see with always like GitHub’s always trying to do. A lot of time is going, yes, we may have something out of the box, but we’ll work with other tools no matter what.

Karissa Breen [00:37:35]:
So do you sort of see people moving away then from. And I feel like I’ve spoken about this a lot like point solutions then. And we are getting the single pane of glass. Interoperability, observability, like we’re trying to do that. But then I’ve often heard from people as well that even if, depending on the tools, if they’re trying to integrate, there’s still a little bit of, to use my bad phraseology, like clunkiness still, there still could potentially be a gap there. What are your thoughts on that?

Craig Wilson [00:37:57]:
Sometimes, like, you want either the vendor, you’ve got to fill the gap, but sometimes even constructs themselves across too many things. And so they don’t do everything well, they just do it just enough. That’s why, you know, there was always that thing in the past about best of breathe and then everyone then came back to all in one. But now I think maybe we’re in this halfway point now again, where maybe all in one’s not always the answer. And you want to do best of breed, but sometimes the best of breed doesn’t always work. So now you’re trying to work out how to bring these two together and you’re also fighting against business requirements. Like, we’ve, you know, you get a lot going. Cool.

Craig Wilson [00:38:28]:
We’ve got these two tools. This one might be really good, but we’re getting this one for free. Or we’ve already got this as part of all in one solution. So can we not use that one and not use this other one and save money? Always looking at the bottom line, how can we be as lean as possible in operating? But you know, that sometimes, unfortunately comes at the cost of like, yes, you can be lean, but you may then end up having blind spots because the tool is not actually adequately doing what you expect. And so, you know, you could then get caught out by missing, missing, you know, a vulnerable package that came through the system because this tool that was free wasn’t doing a great, great, great job of keeping up to date and doing the creation on it.

Karissa Breen [00:39:06]:
And yeah, to your point there, and I know the start of the interview actually about like technical debt, so release cycle. So how do you sort of, then what would be your thoughts on business requirements operating lean, maybe go to free tool, maybe it’s sometimes dodged, unsure. But then, you know, we gather all of the things that really slow the business down. So how to, how do you find that middle ground in order to keep up business requirements, keep up from, you know, this competitive world nowadays, even from a consumer perspective, like, people want things faster more than ever before. And there is a lot more pressure now on businesses to add all these features and functions within the day. So how do you balance that though, without overlooking perhaps vulnerabilities or without, you know, it being super expensive and all the things that we’ve already touched on. I’m just curious to hear your thoughts then on that.

Craig Wilson [00:39:51]:
It’s a real tough one. You just pulled in so many different directions all the time, you know, you might think you got a solution to a problem that you really think is going to help streamline things, but then the business like, nah, don’t have the budget for it, or sometimes you have to compromise, go, look, I know that maybe this tool’s got. Is not probably not the best, but at least gives us something for now. We can hopefully circle back to it later and reassess it.

Karissa Breen [00:40:13]:
But then you never circle back because.

Craig Wilson [00:40:15]:
You see it ever circles back. And sometimes it seems to be. Sometimes you put off a decision going, nah, this is going to be too hard to implement. You know, we got to get all the engineers to change all their tooling, move to a whole completely different platform. It’s going to be a real pain. And I mean, and sometimes it takes a lot of effort and it does cause a massive disruption. And then you got to hope that you’re actually giving them value at the end of the day, otherwise, what’s the point? Then other times where it like, Jay, for a while we put it off for a bit and then it just, it went pretty quick. Now we’ve got better access to latest features and moving forward faster.

Craig Wilson [00:40:49]:
But yeah, I mean, I think, I think depending, also depending on the business too. Like in our business we’ve got a lot of development teams and using a lot of different languages and tools and stuff like that. So it makes it even more difficult to try and simplify and get things moving faster when you’re having to deal with so many permutations. I’ll say a lot of business you’ve got to try and simplify what you’re trying to do and invest in that simplification and the tooling around that in order to move faster in what you’re doing, don’t try and spread yourself too thin.

Karissa Breen [00:41:16]:
So going back to business requirements and this is always a tough one as well. So, for example, I’m going to give you a bad example. So if it’s like, I’m not a baker, but if I were like, hey, I’ve got to make like a wedding cake and then you go and you get a quote like, well, that’s really expensive because you don’t know what’s involved. But you don’t know what you don’t know, right? So. And there’s so many processes, there’s so many things within the baker person’s like, wow, this is how it goes. Do you think the same thing sort of applies? So when the business is like, hey, we just need this, like, small little thing to be done and you’re like, well, that’s going to take like eight months. And like, really? I thought it was a super small. How do you get to that point where you close that gap in terms of business requirements, but also get them to understand that the little thing that you want isn’t quite as little as you think? And then obviously the frustrations start to creep in on, oh, but I thought it was, how do you start to do that in terms of, we’re going to meet you in the middle by trying to do the thing may result in potentially other issues in other areas, but you kind of want the thing.

Karissa Breen [00:42:13]:
How do you manage those conversations internally?

Craig Wilson [00:42:15]:
I think you’ve got to try and explain all the important things that need to happen and why that’s going to take so long. And sometimes you got to put in business perspective, like, what is the consequences of not doing those things and the potential of that consequence coming up if you don’t do those things and then unfortunately means, yeah, then they have to make that decision on what is important right now and then they can make that decision to either streamline or cut things out or revisit them later. Because I know that, yeah, in the day, you know, you want to do the perfect thing. Like, I’ve been through projects where you, you want to do it perfectly and you get to, you get into production, it’s like, hang on, but we haven’t done any monitoring. How are you monitoring this thing? And it becomes an afterthought. But, you know, really should have been done earlier and that would have made it easy to manage. But the business, like, oh, what? I need it out the door by this day. It needs to be in the customer’s hands by this day.

Craig Wilson [00:43:00]:
So you’ve got to start to take on a little bit of risk. So you need to understand and assess your risks and have them and even have them captured so that you can make sure that they are revisited and triaged after the fact so they’re not lost. Because sometimes you make decision, cool, we’re not going to do this. We’re not going to do our vulnerabilities package scanning, because we haven’t got time for that. We’ll do it later. But then everyone forgets and 12 months later, you still haven’t circled back to that project. But you need things like a risk register to make sure that’s all highlighted so that everyone’s well informed about what’s going on.

Karissa Breen [00:43:30]:
So I want to sort of shift gears now for a moment and start to look ahead on, obviously We’ve spoken a little bit about migration, why companies need to move more towards operating faster because of business requirements, also for customers, competitive landscape at the moment. What about, though, in terms of Iris’s view or your view more specifically, Craig, around, you know, innovation on the platform, automation, all the things that, you know, people in the industry and the community is talking about. Do you have any sort of insight on that?

Craig Wilson [00:43:56]:
Yeah, I mean, obviously AI is a big thing at the moment. The big thing at the moment, you kind of feel like it’s that fomo. You kind of feel like you’re missing out if you’re not doing any of it or doing anything. So whether you’re using it to assist in your development lifecycle or you’re embedding it in your products in order to provide value to your customers. But really the key thing is trying to figure out where is it most important. Because you can shove new technology into everything, but you’ve got to make sure you’re adding it into the right spots, that it’s actually doing the right thing. Otherwise it’s just, you know, it becomes probably a feature that’s not utilized correctly. So one of the biggest things we’re going to spend time with at the moment is, you know, a lot of developer tooling, they all want to use AI, they want to use for their coding.

Craig Wilson [00:44:38]:
So, you know, we’ve already provided, we’ve already provided things like GitHub, Copilot, but you know, there’s a lot of other new things coming out, like Cursor, where it’s a fork of VS code, where it’s in the AI is all embedded into the tool. It’s like that. Hey, but how does that compare to using Copilot with the agent mode? Like, what is the difference? Like, why would we use that over the other? But then someone else comes in, I want to use another one called Client. So that’s another AI tool. And that one has other issues because that one, you know, that one doesn’t lean on a vendor to do the processing. You know, it just leans on you hooking up your own. But then you’ve got to hook that up and then what’s the security around that? Is that another procurement? So, you know, we want to provide all this stuff to make help engineers go faster so they can keep up. Because obviously, yeah, if you fall behind, then you’re going to.

Craig Wilson [00:45:25]:
You’ll start losing customers and you’ll start. Someone else will be doing a better job of it. But also you’re trying to get that AI into The tooling, getting AI tooling also means you need a lot of knowledge. You’ve got to get a lot of data out of your, your systems, make sure you’re getting it out also securely making sure you don’t have any leak around that. So you’ve got to get good at managing data. And that’s where you know, you’ve got to have a good data lake or a good policies and procedures around that and people to managing governor to me, that’s the first part. And then you go from like managing your data is the first thing. Then you can start going on top of that and go, okay, how are we going to surface this data and how are we going to get it into the client’s hands? Yeah, I was just thinking like we’re seeing a lot of, like a lot of us know a lot of assistance with AI can help streamline things like we’re in the market of wealth management and trading, things like that.

Craig Wilson [00:46:09]:
And you can see where that can help with financial advisors and things like that, where that instead of them having to manually go through them, create a portfolio, you can embed, they can embed their own knowledge into a model and then use that to generate guidance for their customers that is based on their views but can be automated and sent out a lot faster than dealing with a customer, customer, customer. Then they just have to, you know, they can just take that as a, as a starting point and actually go and adjust it and build on it. Like, you know, I’ve always said AI is like, it’s like speaking to like a million people at once and getting one answer out of it, right? So you’ve got this wealth of knowledge. So instead of going out and doing research and find different viewpoints and then you’re trying to figure out of all that what’s important. You’re just asking it a question and it’s trying to create all that, bring all the information together and say here’s an answer, right? But then what I find generally is you can ask that same question again and you will get a slightly different answer because just how that heuristics went through and prioritized the different parts of the information. You really got to be careful that it’s not always going to provide an exact truth. You’ve got to still use your own knowledge in order to understand whether that is something that is appropriate or whether that is, is something you, it’s correct to what you’re trying to achieve or whether it’s something that you need to actually, you know, either ask Again or adjust to what you actually want. So you’ve got to be really careful.

Craig Wilson [00:47:27]:
So I’ve heard here a lot about agents, things that, where agents are going to be running everything. He just, the agent’s going to make decisions and do this and do that. And look, it does make sense. And maybe if, if the, if the model’s tuned up in such a way that you can have a high, high trust in the outcomes of that decision because it’s, you know, maybe it’s more finely scoped, then that would be great. But I could also be a bit weary that maybe it’s going to start making decisions that maybe are not correct. Like, I mean, I’ve, we’ve already had a situation where we, for a vendor, I won’t say at a vendor, we asked them for some help and then they came back with an answer to try. And so we tried it and we realized, no, it doesn’t work. And then we realized that they had used AI to generate that answer and that answer was incorrect.

Craig Wilson [00:48:06]:
So yeah, you got to be careful.

Karissa Breen [00:48:08]:
Okay, this is interesting. Do you think that people are going to get to the stage where they’re fully like, oh, I’m just going to kick back, as us Australians would say, and just let the agen run in the background with making decisions? I mean, it’s still quite early days for that sort of stage, wouldn’t you say?

Craig Wilson [00:48:23]:
I would say so, but there’s some people I’ve bumped into like launches and things where they, from companies where they, they believe that’s, that’s the thing that’s going to happen. Agents are going to run everything. We just tell, give the agents all the information and they will just do everything for you. I’m like, but that sounds, even really, that sounds dangerous. Like, so this agent’s going to be able to, you know, authenticate in a whole bunch of critical systems and be able to do things that you would usually put a lot of barriers and trust around. Like, is that what we really want? Like, is that what’s, what’s happening? Like thinking maybe in isolation it might work. But you know, in a system where you just give it free run to a whole bunch of other systems, you know, imagine like instead of me having to deal with numerous departments at work, I could just ask it, I’ve asked it a question, you know, can you go fix this up for me or I don’t have access to the system, can you provide access? And either it would just make the wrong decision, go, yeah, no worries, and just go and change all the tooling to add me as someone who’s got access and then I’m in and it’s all good. But.

Craig Wilson [00:49:14]:
But were the right gates in place to make sure that, you know, to make sure that. To look at the fact that, hey, maybe I’m not in the right level of clearance to actually get access to this system and I really shouldn’t be provided with that. But, you know, we’re trying to do it because we’re trying to move fast and we’re trying to give people’s times back to do other more important things. But, you know, how do we do that?

Karissa Breen [00:49:31]:
So just on that for a moment, if that’s the case, does that mean we completely, like, eradicate like attestations then? Because we’re not checking or anything?

Craig Wilson [00:49:38]:
I think that’s because it’s more important to have attestations. It’s more important to have something to.

Karissa Breen [00:49:42]:
Well, that’s what I was just thinking as you were talking. But you think people will just forego that?

Craig Wilson [00:49:46]:
Like, oh, I don’t really know. It’s just. I suppose it’s the worry of where you don’t think through the consequences of what’s happening. Like, there’s already. I think there’s already that case recently where someone talked about where someone worked out that I can email someone a prompt that someone would be accidentally, would be fought, will be run against their AI environment and then siphon data out because it would just respond. And so they figured out how to siphon data out. There’s an avenue there that hasn’t been thought through that it sounds all great that I can maybe email myself with a prompt and get a response, but now someone else can do that.

Karissa Breen [00:50:18]:
So would you still say that we’re in this? I probably know the answer to this. But like what you’re saying, like, yes, it’s good we should be able to do these things so we can do other things in critical thinking and all the strategic thinking. But then with that opens up another can of worms on what you just said. So do you think we’re still in this territory of uncharted borders that, yeah, we kind of know what we’re dealing with, but do we really? Do you think it’s going to take a fair few more years to really understand the fidelity of what we’re dealing with here or. But every day, I mean, I mean, media and something’s coming out every day is even having me to keep up. So what does that sort of look like then as like sort of the final comments and Close, you know, closing comments Final thoughts that you, you can think of.

Craig Wilson [00:50:56]:
I think definitely it might take a long, a bit longer for us to fully understand the avenues of what this tool is going to open up. Like, I mean, another example I’ve told people work like we’re looking, have having a, you know, from a business perspective, we’ve got data in different systems and, you know, it’s hard to search across them. Some of the searches are not great. Just want a tool where I can just ask a question and bring all the information up to the surface and give me an answer like, of like what you’re asking, what is someone’s view around a project or something? Or what’s the status of a project? Or I said to someone I could ask the question, what have people been talking about me at work through like Slack or through social channels and things like that. And then it can surface that up really quickly. So what sounds like it’s a really cool tool. I can find business information. I can now use it to find where people are maybe talking negatively about me and that has negative common connotations that I probably, probably don’t want to be doing.

Craig Wilson [00:51:43]:
I mean, understandably, people said, you know, as long as things are in private chats, then, you know, the idea is to not surface that information. But, you know, if it’s in a public channel, it could be used in a, in a way that you don’t expect. So that’s what I mean by, you know, it all sounds great. I mean, every tool always has a positive and negative aspect to it and so do we, do we understand how this is all going to work? I think we’re still in that early inflection point of everyone’s jumping on AI tooling. And yeah, it’s going to take several years before we probably hit a point of more stability and knowing what we should be doing, what we shouldn’t be doing, and new tooling to come in place to help govern those aspects.

Karissa Breen [00:52:27]:
Joining me now in person is Tal Sifati, architect lead at JFROG Security. And today we’re discussing how the CISA MITRE CVE funding scares it out. Thanks for joining me and welcome.

Tal Zarfati [00:52:38]:
Thank you for having me.

Karissa Breen [00:52:39]:
Okay, so there’s a lot of questions. When I was writing your interview, I had so many thoughts in my mind, so I’m hoping we get through majority of the content, but maybe let’s start with perhaps people who aren’t aware that sort of the MITRE funding scare. Walk me through quickly what happened and then walk me through as well. Why is that a massive risk at the moment?

Tal Zarfati [00:52:59]:
Okay, so what happened is due to some political intrigues in the United States, Mitra, which is the official organization that is in charge on managing CVEs. CVEs are common vulnerability numerators. That is, it’s the standard today, the open standard for managing software vulnerabilities. On. It’s not just public. On every software there is, if a vulnerability is known, a security vulnerability is known about any kind of software. It’s currently being managed by this organization. And this organization had been funded primarily by the United States government.

Tal Zarfati [00:53:35]:
The political turmoil in the last year, there was a risk for defunding them. So what they’ve done is they’ve issued, it was a letter to everybody that said that there is a risk that they will stop their activities. And the reason that this is such a major issue is that for the past, if I remember correctly, about 15 years, they became the de facto standard about managing all of those vulnerabilities in the world. So the risk was that the moment that the carpet was pulled from under them, nobody will have any centralized information about software vulnerabilities or new software vulnerabilities.

Karissa Breen [00:54:10]:
Yeah, so this is super interesting. Okay, so you mentioned before the carpet being pulled from underneath them. Do you think that nobody thought about this could happen? So it’s like, for example, electricity, you don’t really think about the light switch until it goes out, right. And you’re like, oh, I really need it. Do you think that was people’s mindset, though, when it’s like, okay, we have all this access and with this repository to all this good information in terms of intelligence, and then we don’t have it anymore?

Tal Zarfati [00:54:31]:
I think that’s largely the case. And to think about how this industry has evolved, Think about it like, about 10 years ago, the world of supply chain security was still in its diapers. So the ability to track and understand what vulnerabilities are out there was growing. And if I remember correctly, 10 years ago, there were a lot of companies were managing those kind of vulnerability information, vulnerability insights on their own, and they were selling it. And then this operation, as a government for the public environment started to grow traction, and more and more solutions and more and more security practices have started to rely on that without really managing the risk that it will someday be automagically gone. Mostly, I think it was a reliance on the system. And in recent years, I’ve been in such a turmoil, this thing caught a lot of people by surprise.

Karissa Breen [00:55:27]:
Interesting. Okay, so we saw that Reliance. So do you think that there’s this stigma in the industry and you can focus on this for a moment. There’s this over Reliance then. So it’s like, well, I’m just gonna keep using like we just mentioned this repository to get this information. It’s not there and then when it’s not there, there’s a gap.

Tal Zarfati [00:55:41]:
I’m not sure that the world over Reliance is is the correct word to describe that. It’s an ecosystem that has been growing. The challenges behind MITRE and NVD as a whole have been growing up in the recent years. Regardless of this risk, if you will see, there are a lot of other vulnerability databases that have been growing, each with its own story behind it. You have the GitHub security advisory which is largely parallel to that, mostly focused on open source packages. If MITRE and NVD are covering everything, GitHub Security Advisory is covering a specific amount of software packages. You also have advisories per ecosystem. For instance, NPM as the Node JS package manager, PyPi as the Python package manager, GO as the GO package manager, have their own.

Tal Zarfati [00:56:34]:
Google as a commercial company is also building their own aggregation of this kind of information. JFrog is building their own aggregation of that information. Because the industry has grown to learn that that single source of information, as much as everybody started to rely on it, had its challenges. I’m not sure that the challenge that was on top of everybody’s mind was that one day it will be gone other than how we can rely on the quality of that information and how is the quality of that information can work in our specific cases, but the reliance on that specific environment has been changed for the last couple of years at least. I know that in one of the things that we’ve been doing is we’ve been starting to address the fact that the CVSS scoring system in NVD is not enough for user for us to really assess the scoring system or really assess the criticality of a vulnerability. And we wanted to provide our own enhancement or our own enrichment to that kind of information and have known that others have taken the same route.

Karissa Breen [00:57:40]:
So Tal, the reason why I asked you that question was more so I understand like the adjacent sort of repositories, but it’s more like they were sort of the original one that people, it’s more well known been around in terms of tenure, if I’m correct. So that was sort of more my view in terms of the over reliance piece. So that being said, moving from that or sort of jumping to the next thing would be just say it goes offline. Do you think people will just go to these other ancillary sort of.

Tal Zarfati [00:58:05]:
I think that’s what people do. I know that a lot of solutions, a lot of security solutions are going in the path of aggregation. That is, I will go to specific advisories, I will aggregate from different advisories and I will provide my own value on top of that. Or I will choose and pick and choose which environment or which advisory works best for me in the context of what I’m willing to choose.

Karissa Breen [00:58:30]:
Do you think that becomes decision fatigue then?

Tal Zarfati [00:58:34]:
That’s a good question. I think that users themselves will rely on third party products to provide them with that information and I think that a lot of businesses will optimize on that. We provide you, we save you from the challenge of making that decision and we make the decision for you. For instance, if you look on how operating system vendors are doing that, for instance, Red Hat or Debian or Canonical for Ubuntu, they do just that, they do the aggregation and they’re providing their own advisory for their customers.

Karissa Breen [00:59:07]:
So the reason why I think this is an important question is because you’re going to have people, depending on their background, experience, what they’re more familiar with, they’re going to go to their standard, well, I use this source over that one. So how do you think people get more of a holistic view on this intelligence to make better informed decisions, have better like understanding of what’s happening? Or do you think it’s all of the things but then that requires, like I said before, the decision fatigue, et cetera?

Tal Zarfati [00:59:34]:
I think that security products and security product companies will become the de facto standard for those either be it like companies that will publicize this data, such as Google companies, that this data will be part of their IP or part of the value proposition that they give to the customers.

Karissa Breen [00:59:53]:
So it’s a fair assumption that that’s where you believe the industry will start to head down.

Tal Zarfati [00:59:58]:
I think the industry is already heading down that term and there are several companies and for included that provide their own aggregate layer of information. And this is the value proposition that they give to the customers.

Karissa Breen [01:00:09]:
And so therefore it probably wouldn’t matter then if like an open source sort of repository, if something were to happen because they can rely on, I think.

Tal Zarfati [01:00:17]:
That’S part of the selling point of that kind of solutions.

Karissa Breen [01:00:20]:
Got it? Okay. Okay. So therefore, with that being said, do you think that these other repositories will become sort of maybe a bit obsolete, redundant then moving forward Based on what you’re sort of saying, like what I’m thinking through in my head now you’re.

Tal Zarfati [01:00:34]:
Speaking, I find it hard to say anything about an organization such as mitre. I do think that other security advisories will start to consolidate. That is you’ll have the ecosystem related or the environment or the specialty related vulnerability advisory and you’ll have the aggregators. Whether you have like a second of aggregators that are aligned with specific product and not with a specific source. I assume that this is where the industry will end up because I can see that happening right now. I’m not sure how it will affect like the common for public environments.

Karissa Breen [01:01:11]:
So what would you say in your experience would be a risk of people or user sort of just solely relying off a single source of trees? Do you think it is better now that there is other options, other feeds, etc. That people should get more well rounded?

Tal Zarfati [01:01:28]:
I think it’s better that we have more fees. I think that part of the challenge that we have more feed is that the loss of certainty in a single source of truth is the fact that the less common environments, the less common products, the less common challenges won’t get that level of information. For instance, if we look at, I don’t know, GitHub security advisory, it’s very broad and in a very high quality and they cover a limited amount of environments and limited amount of ecosystems. They are optimized over libraries, in public package managers. But things that are not aligned with that ecosystems are not covered. And the challenge of what will happen to get that information is an unresolved question. Again, I’m not sure where the industry will go. If I need to bet on something, I think it will be relying on the fact that the protocols and the standardizations on how to exchange this kind of information is picking up traction and that would allow for easier curation and aggregation of that information from even small actors.

Karissa Breen [01:02:34]:
Do you also think standing on that things aren’t covered, but also would you say the fidelity just isn’t there on certain things as well? Perhaps it’s not a lot of debt?

Tal Zarfati [01:02:41]:
Absolutely, absolutely. The fidelity is not there yet. But that was part of the challenge with Nvidia as well. If we look on Nvidia, if we look at before the MITRE scare, if you look at the past year, 2024, NVD had also challenges there and they stopped delivering assessment for vulnerabilities. So for the better part of 2024, vulnerabilities didn’t get CVSS scores from NVD. And then products who rely on disability know that we have a vulnerability. But understanding what’s the risk there and what’s the severity of there, security organizations had to rely on other sources for that information as well. And I think this is also something that drove, at least for the past year, the need to broaden the spectrum on how and where you get that information from.

Tal Zarfati [01:03:31]:
That’s what we did to all of our customers. We have our own catalog product which does that aggregation about all software packages and all vulnerabilities. And we’ve done just that. We’ve aggregated vulnerability information from different sources and different advisories and we enrich that with our own security information for security.

Karissa Breen [01:03:49]:
So what I’m curious to understand now is everything that you’re discussing, if we just zoom out, just say that there’s delayed CVE intelligence, what do you think that actually then looks like for companies? But then also that attacker opportunity in terms of that window.

Tal Zarfati [01:04:04]:
So the attacker window here is, I think the answer is in the body of the question. The time that it will take for potential victims to bridge the gap between a vulnerabilities being published, for instance, as it exists as it was discovered, but the information on how to mitigate it, on the severity of what it affects is not there. It allows potential attackers to have a head start on their potential victims by getting that information. It’s resembling the atmosphere that we had 20 years ago when Microsoft had a patch Tuesday. And every Tuesday they just discover the patch and then every hacker in the world just try to reverse engineer that patch extremely fast. So you’d be able to get an exploit before, before everybody has a chance and to install that patch and to protect himself against that. So this, this is the potential. And I think that this is where security companies such as JFrog has stepped in and offer their customers the ability to get the same experience.

Tal Zarfati [01:05:05]:
That is the security research teams in those security companies try to track that information sooner rather than later. By the way, not just security companies, vendors for operating systems, again such as Red Hat and Debian and Ubuntu, have employed their own red teams to do this analysis and provide their own severity and information about their own packages without relying on NVD anymore.

Karissa Breen [01:05:25]:
Okay, that’s really interesting. So then would you say, and I think you sort of described just before then with attackers, the velocity we’re attacking the AI depending with AI, all these things in terms of velocity there is there nowadays and it used to be so do you even think that, you know, cybercriminals even weaponizing Even the smallest sort of vulnerabilities to because of the delay in intelligence, of course, but then also just sheer volume with what’s happening with AI.

Tal Zarfati [01:05:53]:
That’s the case regardless of the Metro situation. Weaponizing of vulnerabilities has been the name of the game in supply chain security for the past 2025, for the past 15 years at least.

Karissa Breen [01:06:04]:
Yeah, but with that in terms of like the volume of it now, obviously we’ve seen, we’ve seen an increase, we’ve seen things faster than before. Like is that sort of.

Tal Zarfati [01:06:12]:
The answer is definitely yes, because vulnerabilities have been weaponized for a long time. SolarWinds attack, which I think was like the first big major software supply chain breach, was about 12 years ago if I remember correctly. And ever since then those angles have been exploited. And the more this volume grows and the more attack vectors are being found, the bigger the leverage of that by attackers. And if you go to the world of agentic AI and developers that can open new verticals for supply chain attacks, the volume of that will go. If we look at the talk I just gave and we can see that the statistics that we cover over a hugging face that we saw that the number of machine learning models grew from last year to this year by three times, but the number of incriminated malicious models grew seven times.

Karissa Breen [01:07:07]:
So maybe can you share a little bit more high level or summarized version of what you just discussed on stage?

Tal Zarfati [01:07:13]:
Sure. So I think that the major point I wanted to show on case was more code is being written, more software is being written, more things are considered as software. That means that the volume and the attack surface grows tremendously. The world that’s being opened by AI, by machine learning, that is agents writing code and machine learning models as packages themselves expanding this attack surface on the supply chain and the fact that these packages or these models or these new type of software is still not fully aligned with all the security best practices and methodologies and organizations opening up these organizations to standard supply chain attacks. So this was the base promise of my talk. And then I dive a little bit into the research that we’ve done over the sheer volume of malicious pack malicious models that we’ve seen in hugging face and the fact that because this field is still fairly new, it’s easy for a lot of security assessment tools to just check the box and to say anything that is suspicious is something that you need to avoid and be careful. But that is a little bit contradicts the velocity of adoption of this capability. And the correct solution like with any other new technology is to build better and deeper analysis tools for that.

Tal Zarfati [01:08:48]:
What we did is we implemented capabilities from the world of static code analysis and we used that to incriminate suspicious in machine learning models. And then I shared some statistics about the research that we’ve done with Hugging Face. We’ve found in Hugging Face that other tools currently mark about 4,000, 4,700 models as as malicious. But our number after deep diving and manually analyzing those and then automating that, that analysis came down to about 171. Which means that 90%, 96% of the models that are being marked as malicious by other tools are false positives. And this might, and I’m extremely sensitive to that, cause a lot of distrust in the process because if I can’t use if me as a developer, me as a machine learning engineer, as a data scientist get the response that every model that I want to onboard to my system is suspicious. And very quickly I can see that most of the cases are false positives. It will make me lose trust in the tools, make me lose trust in the process.

Tal Zarfati [01:09:58]:
And one or the other will happen. Either I will stop onboarding AI models or my organization will have a lot of troubles to do that. And I don’t think that that will be the case because we are living in a world where onboarding AI is a must. But that will happen is the other side of this. And that’s alert fatigue. People will start ignoring the alerts. So the only thing that needs to be done here, and this is our approach to it, is to implement and make the effort to do deeper analysis on those models that are suspicious, to incriminate them as malicious with intent.

Karissa Breen [01:10:29]:
So how do people sort of correct, will they have to retrain the models then? Is that what you sort of saying?

Tal Zarfati [01:10:33]:
I’m not sure what you mean by retry the models. The information is how do they get information whether a model is malicious or not. So there are tools to do those scans. JFrog environment JFrog platforms does those scans and can give you that information. But Hugging Face is a central repository for that as integrations that will show this information. And JFrog and HuggingFace are partners and you can get this information that provided by JFrog through the hugging face web UI. That is if you go to Hugging Face and go into a model, you’d see what the verdict from JFrog, what the verdict is for other vendors. And the main claim here is that we believe that our answer there is much more precise and accurate and reduces the amount of false positives by a very large amount.

Karissa Breen [01:11:21]:
So I want to sort of shift for a moment and talk about companies perhaps how they triage vulnerabilities internally. And I’ve experienced this myself internally, when you’ve got thousands of them that stay there for years with everything that you sort of discussed me today in terms of like new attack surface, because more code’s been written, developed, all of the things. If we were to sort of look at it from your perspective, given your experience, what do you think needs to change then in terms of the reality and where we’re moving towards now as an industry?

Tal Zarfati [01:11:50]:
So I think that there are three very important things. First of all is the contextuality of the results. The biggest part, and it’s not just with malicious model, it’s just our latest research. But the biggest problem with the sheer volume of vulnerabilities, the first thing is the fact that a lot of them are considered, at least by developers, as false positives. Mostly because to understand the correct context of that appearance is challenging. And there are tools, jfog offers those to do that type of contextual analysis and vulnerability prioritization, and this is the biggest trend right now in the world of vulnerability management, is how you prioritize and reduce the noise from vulnerabilities. And I think that’s the first step and the second step is automating governance. That is, once we can trust that the vulnerabilities that we get from those tools, the ones that matter and the ones that the severity score reflects the blast radius of those vulnerabilities in my context, in my environment, in my organization, the next thing I need to automate the governance.

Tal Zarfati [01:12:53]:
I need to make sure that I can put in the gates inside my software development lifecycle, inside my application management lifecycle that allow me to prioritize what I need to fix and when I need to fix it. For instance, if I have an application that’s in development lifecycle, and I know that I’m running a lot of test iterations on that, fixing a medium vulnerability is not that critical in that point. It would be critical when I want to promote it into beta testing stage or point to staging or to production, depending on my development lifecycle stages in my organization.

Karissa Breen [01:13:27]:
Just before we go into that a little bit more. So going into the contextualizing of the vulnerabilities. So would you say. So this is where I think there’s this conundrum with like different teams. I’ve gone to these meetings, say, hey, this is. We’ve looked at the application, we’ve pen tested it, this is what we’re dealing with. Then obviously people have got different agendas to hey, I need this project to go live or I’m KPI’d on doing this feature and function by this day. So therefore you start to get into these arguments around security point of view.

Karissa Breen [01:13:57]:
We think this, you’re thinking there’s some tech risk, business risk, systems owner, this domain owner, all these other people you got in the room. How do you get to the point where you’re on the sort of same page where you’re like, okay, we all agree or we, you know, half agree that this is where the priority is?

Tal Zarfati [01:14:12]:
I think it is leaning on two things or three. First of all, again, the moment that you have less items to argue, the easier is to get to. Let’s call it, as you call it, a half agreement. If I’m arguing with you about 1000 items that you ask me to as a developer to go and fix, it will be a different kind of discussion than if you ask me to fix one thing, that’s one leg of it at the moment that I can prioritize what I need to do and focus on what matters and has the biggest blast radius, then the easier it will be for me to accept that. The second thing in that case is explainability is to explain why I decided these are the ones that has their impact and the other things are not critical. And the third thing is something I haven’t mentioned before, but I think it’s also part of the next step that this process will go to is give that information to the developer as soon as possible. Mostly because if I’m a developer, my agenda is to ship fastest or to ship as fast as I can. I am getting limitations, for instance, on certain packages to not even use them before I’m starting to onboard a new package.

Tal Zarfati [01:15:20]:
Some packages will be excluded for me because they do not comply with the policy or they’re not secure enough. And then I will find the vulnerabilities on those packages during the development lifecycle and be able to avoid using them, replace them or fix them before I go through all the process before I too much vested in it. If the remediation suggestion I will get from my tools will be more accurate for me. For instance, one of the biggest challenges in this world is okay, in order to fix this vulnerability, I need to go and change, I don’t know, four transitive dependencies. But I can change transitive dependencies because I’m dependent on one package that brought those packages and this change is not applicable or not actionable from my perspective or the recommendation that I get require me to change the way that I use the library. Those are all challenges for me as a developer to address those. That is, the easier for me as a developer to address the issues, the easier for me it will be to say, you know what, I’ll do that. The sooner I get that recommendation, the easier for me to even avoid this kind of discussion.

Karissa Breen [01:16:28]:
So then, just one other question, a couple of things that I’ve got as well. So would you say historically, I mean, even in my own experience, perhaps we didn’t to your point, contextualize in terms of the environment. So I’m used to working a bank, financial services, heavily regulated, lots of other issues as well. But do you think that perhaps people aren’t explaining it in like because of this company we work in, like this is why this is an issue and people are just too general about it and people therefore don’t understand.

Tal Zarfati [01:16:56]:
I think that in those kind of environments the regulation makes it easier and to, let’s call it, use authority in order to avoid the technical discussion. You need to do that because that’s the requirement I have from the bank. And you can’t argue with me because you are working in a bank and you need to do the regulations because I’m compliant.

Karissa Breen [01:17:18]:
And it’s the only thing that gets people offside though. So you take compliance, regulation, governance, people start to lose interest.

Tal Zarfati [01:17:24]:
Well, that depends on the people. I think that people work there that has to face those kind of regulations. This kind of governance and this kind of assistance with complying with those regulations reduces a lot of restriction. So the feedbacks that I’m familiar with regarding that are of excitement because these kind of tools, these kind of capabilities will save me the trouble on having to face those regulations. It will make life easier for me because I wanted to deal with the bureaucracy and with the authority and with the complexity. If you cut the talk and the tally head just before me. So Ollie there from I can’t remember the name of the company he works with. He was excited about the concept of what Shlomi called the Dev Gov Ops.

Karissa Breen [01:18:07]:
It’s just more so historically would be security people would run it developers in a way where it’s like I’m telling you what to do like a police. And I think that as you would understand gets people offside. So historically as a security industry we haven’t done a great job at trying to integrate people. You know, it’s got SEC DevOps and all that sort of stuff. But I’m saying that maybe there is a little bit of that stigma there to be like, oh great, the security person look over my shoulder trying to tell me I do my job rather than assisting them.

Tal Zarfati [01:18:36]:
Totally agree. Because I think that historically, as you said, that’s the way that the industry goes. But I think that modern technology helps bring those, those barriers down. The industry itself, the software industry itself is becoming less siloed and is becoming more heterogeneous from that perspective. And I think that gets developers, for instance, be more excited about security capabilities and security features and it gets security products and being more targeted to the developer experience. And I think that once those two approaches coming together, the result of it is developers that feel more empowered and more responsibility and sense of ownership on the security perspective of what they do.

Karissa Breen [01:19:20]:
So Tal, do you have any sort of closing comments or final thoughts?

Tal Zarfati [01:19:24]:
We are living in extremely exciting times. I think that the world is changing. I like to say that we live in the future because stuff that a couple of years ago were science fiction are are the reality today. And the outcome of that is that software is more commoditized, more people are doing software and on one instance it’s exciting because everything is software and there are so many ideas and the technology today reduces or reduces the distance between idea and a product. But I do think that the security industry will have to find a way to cover all that because the more code being written, either by people assisted by AI or either by AI, the more issues that can arise and the more of that more that this volume is being, increasing the ability to govern it and to cover it and to review it manually will reduce over time. We will need to rely more on automations, on governance, more on automatic gating.

Karissa Breen [01:20:40]:
And there you have it. This is KB on the go. Stay tuned for more.