KB [00:00:10]:
What’s up, everyone? It’s KB and I’m on the go. At ISACA’s North American Conference in Las Vegas this week, some of the biggest conversations shaping networking, AI and cyber security are all happening at once. This week is where operators, engineers and executives get into the detail of what’s working, what’s not, and what’s actually changing inside enterprise environments at scale. We’re hearing a lot about AI driven networking, automation and visibility, but the real question is how does that translate into security outcomes, operational resilience, and ultimately cost? Because behind every intelligent network claim, there’s a small human team trying to make sense of complexity. Across the next few segments, we’ll be speaking with leaders on the ground unpacking what’s real, what’s noise, and where the industry is genuinely heading next. This is KB on the go from Las Vegas. Let’s get into it.

KB [00:01:18]:
Joining me now in person is Sushila Nair, CEO at Cybernetic and President of ISACA Greater Washington D.C. chapter. And today we’re discussing AI isn’t delivering the ROI people have been expecting. So, Sushila, thanks for joining me and welcome.

Sushila [00:01:25]:
Thanks so much. Thanks for inviting me.

KB [00:01:28]:
Okay, so I find this quite interesting and I want to get into this a little bit more. But tell me what, what have people been expecting around ROI and AI? What, what’s going on here?

Sushila [00:01:39]:
Yeah, it’s fascinating, isn’t it? You know, AIs kind of come out of nowhere and all the budgets are being soaked up by AI. And you know, when you were looking at return on investment, you know, we just recently ISACA. In fact, just yesterday, ISACA released a AI Pulse where they went out and surveyed folks to really have a look at what the return on investment is. And it came back that AI ROI is really unclear and Most people feel 23% feel like it’s just too early to tell. It’s a complete transformation of people’s organizations. A lot of companies are just getting used to using it and convincing the people within their organization to use it. Still coming up with the business cases. Right.

Sushila [00:02:24]:
And there’s a lot of early kind of plays around that. There’s no doubt about the fact that, you know, return on investment hasn’t been extraordinarily high. However, I don’t know if you saw Gartner Forrester came out with this where they said, you know, last year AI was your friend. You know, you’re having chats with ChatGPT or Claude. Right? But this year, AI has put its hard hat on and it’s Getting down to do work. And it’s moving, you know, from more generative AI, having a chat, chat bots, which we’re still learning really to get the workforce to kind of use. And there’s cultural change there, but we’re seeing some return on investment, but not consistently. And we’re still getting used to all of those business cases and the best use cases around in educating workforce.

Sushila [00:03:14]:
But now where we have AI agents, it isn’t just about having a chat. The AI is going out and doing some work. It’s got its hard hat on and you’re getting it to do things like take customer service calls. Right. Or go through and finish pieces of work. And so, you know, because AI is moving so quickly as we, you know, as we look at this year’s return on investment, this year’s return on investment is going to be so different from next year when we’re looking back at AI with its hard hat on, versus AI being your buddy and your workmate.

KB [00:03:48]:
I interviewed someone literally this week at a another conference and they were saying, like we’re in this time at the moment where there’s like AI sprawl, like we’ve all heard of tool sprawl, but now there’s like AI sprawl. So do you think that perhaps people have just put their little nets into different ponds to see what works? So maybe, perhaps it’s hard to tell. And to your point that 23% do say it is too early to tell just yet, but do you think it’s because people have sort of spread themselves over multiple tools and agents to see what responds or what works for them? Perhaps from a workflow point of view, do you think there’s a bit of that in there as well?

Sushila [00:04:25]:
Yes, I definitely think that. And I think as well that last year people that were looking at AI agents were mostly running in proof of concepts. So we’re not really seeing ROI coming out as heavily because AI agents are really going to tip the balance there this year. One client of mine was telling me they had four AI agents last year in a proof of concept and this year they’re rolling out 140. So four not going to be much ROI, right? 140. You’re starting to see some results there. So I think you are absolutely right. People are.

Sushila [00:04:59]:
At first the excitement was they’re trying to AI everything and really getting understanding that AI, what business problems AI is best at solving has been something that we’ve kind of learned as we’ve gone along, as well as getting the workforce to learn how to work with AI. And then this massive pivot, not just from it being a co pilot, if you like, into being an actual assistant going off with the capability to do tasks for you. And that’s having, I think, a tremendous move. You’re seeing that especially in key industries like developers, for instance. You know developers, their work is being revolutionized by, by AI. And you’re hearing folks like Anthropic saying we don’t see that developers are really going to be required. Right? And so I think we’re on a train looking at the ROI now because it’s not like AI isn’t changing. The sophistication of AI is changing leaps and bounds.

Sushila [00:06:03]:
And so we’ve got a moving target when we talk about ROI. So I think we’re going to see a very different picture next year.

KB [00:06:11]:
And okay, so this is interesting. So if we look at like tools, people experimenting and seeing what makes sense to them in terms of business use cases. If you look at like when OpenAI was like top dog, like now it’s not anymore. So do you envision that it’s sort of anyone’s game? So it’s like it could be for a moment, maybe anthropic, Stop dog, it might not be in six months. I mean, we don’t know. We can’t tell the future just yet. Do you see that there is going to be at one, a lot of competition. It’s going to be intense between the capability with use cases, et cetera.

KB [00:06:45]:
Because if you look at OpenAI, it’s more of a consumer based product, right. Than it is for a business. So do you think that we are going to start to see these players shift, move up and down quite aggressively over the next few months?

Sushila [00:06:59]:
I think there’s a fascinating question and I think two things are happening. Firstly, I think the AI players are working out how they’re going to monetize. You’re hearing talk about putting advertisements in, right? Maybe leading straight over to some kind of shopping site. So integration, let’s say with Instacart, right? Hey, come up with a recipe, place the order. All of that kind of being automated around who gets paid for all of that becomes this interesting question, right? Maybe you’re using ChatGPT to design your order for you and it goes off and it orders it, right. And arranges it all so that monetization becomes a question that they’re still playing with. Much like it took Facebook a while to monetize, right? Or Google to work out all of its monetization around search and of course Google is very much working that out. They’ve had a lot of experience in this area.

Sushila [00:07:50]:
Foundational models are very expensive. So the foundational models like OpenAI’s GPT models or Meta’s model or Claude, they are expensive. And most people, what they’re doing is they’re taking those foundational models and they’re building upon them. So I actually sustain, expect that you’re going to see further concentration. So you’re seeing cloud, everybody put their infrastructure into cloud players. Now you’re going to see people not wanting to build foundational models because they’re so expensive to build and then using that foundational model and then building on top of it. So my prediction would be that we’re going to not see a huge number of players, but we may also see a struggle around the growth of laws around both privacy and data sovereignty playing into the fact that there may be more local players. So Europe may turn around and say we don’t feel comfortable using Claude.

Sushila [00:08:51]:
We need to have European equivalent of Claude, perhaps. So that might also create more foundational models just because they have different laws around privacy and also consumers concerns. We’re seeing this kind of political tensions globally, which often cause organizations to then create different kinds of solutions. So you see for instance Macron coming out saying, oh, we don’t feel comfortable using Microsoft Teams or Zoom, for instance. So will that also cause additional players? I think very much so. But even with all of that, because the cost of building a foundational model is so, so expensive, it will be like the cloud payers, they’ll be focused on the few that can build those foundational models for the millions that it costs?

KB [00:09:41]:
Yeah, that’s a good way of putting it. So are you saying that if you look at like hyperscalers, so let’s talk, look at the Big four. And then people have obviously got their own workflows that they use, maybe they use an independent company to do that, et cetera. So we’re going to start to see more of a proliferation with like AI workloads. So in terms of there’s no point in trying to build it, we can just build on top of it and leverage what they’ve got. Is that where maybe we might start to see a little bit more ROI from independent companies as well? And the reason why I asked that is still a lot of people like, well, where do I start? How do I go about it? What does this mean for my business in terms of use cases or specific needs for my business requirements? So do you think that until we’re starting to find these AI workloads doing some of the work for these organizations, it’s still going to appear like, well, the ROI just isn’t there.

Sushila [00:10:32]:
Yes, I think that we have to move beyond AI just being chatbots to actually being integrated into workflows. And that’s when you really see ROI, I think. And to a greater extent, using a foundational model without customizing it to your environment has limited value. And so if you were just using ChatGPT to answer questions of our work, you can’t upload your client’s information because you’re concerned about. So you’d have to get a corporate access to have a corporate contract with them so that you have some guarantee around privacy. But even with that, you really want to ask questions that are specific to your company workflow. You want to say, hey, at company apc, how do we order pencils? How do we do this? And so that capability, get a foundational model to query your internal intelligence is well nigh possible. And that’s a smart way to do it because it saves you billions from building your own model.

Sushila [00:11:31]:
Right. You leverage a foundational model, but you tell it to use your own internal knowledge when it’s actually coming up with its answers. And so we’re learning all about these kind of methodologies. But even then, if the model is simply answering questions, you’re never going to get the same ROI and as if the model’s doing things for you like create, creating a whole PowerPoint deck and then, you know, emailing it off while you are asleep.

KB [00:12:00]:
Right, right, right, yes. So, okay, so then talking, going back to more the ROI, more specifically, what is it that people are like trying to ascertain? Is it that, well, I want to try to save on cost and yet I’ve put my little feel is out to 50 different AI platforms etc, and therefore I don’t see the ROI yet from a cost point of view because it’s still early days and they haven’t gone all in on one particular tool perhaps or product. Or is it that? Well, actually I was trying to leverage AI to downsize my workforce and yet I haven’t quite seen that. But again, there’s going to be a bit of a handover period. You can’t just bring something in and expect that it’s going to work instantaneously. It’s going to take a bit of time in terms of what that new workflow internally looks like and processes, et cetera. So what would you say if you had to Boil it down to people are looking to really ascertain from the ROI.

Sushila [00:12:56]:
You’re right. What we’ve learned is the use case really matters and the business use case really matters. And so understanding the problems that AI is good at solving also really matters. And as much as we can shift over into AI automating workloads, that’s where you really get cost savings. Right. If you automate, you save money. If you’re not automating, you may have some nice things going on, but you’re not really saving serious money. And so it’s that automation really, that is the difference between the chat bots that we’ve had up to now to where these companies are going, which is AI agents.

Sushila [00:13:36]:
And AI agents are what automate workflows. And really you’re seeing solid use cases around AI agents so that we’re seeing people roll them out. And the moment they start to automate, that’s when you see real ROI. Otherwise what you’re doing is maybe you’ve got nicer slides than you used to, or maybe you’ve got your graphics looking better, or it’s cleaning something up for you. But the moment you start to automate, that is really when you clear see clear return on investments. And there are people that are seeing ROIs. That’s also well worth pointing out. And I think those are the people that were very strategic about their business use cases.

Sushila [00:14:22]:
And as they kind of started to roll it out, they looked at what was working and what was not working. And also with all early technology, it takes a little while for it to mature. And as it matures it gets better and better.

KB [00:14:38]:
Right?

Sushila [00:14:38]:
And then you get more ROI. And I think people are afraid to hang back because they know their competitors are investing as well. And at some point we all know that if you don’t use AI, you won’t be competitive. And so we see this kind of incredible commitment and race because we know the inevitability is that we need AI to be able to do that very vast kind of analytics that we need to really be able to optimize and automate. And it’s automation that saves us money. So those agents are going to be a game changer.

KB [00:15:14]:
Right. Okay. Would you say, generally speaking, we as the industry perhaps have over invested in AI and maybe underinvested in people and skills because still there’s still people that are not going to know where to go. So you’re still going to need some, some physical human person to then go and do the due diligence. Okay. Well, what does this look like? So where does that sort of sit with you? Because yes, whilst I know people want to do more with less nowadays and we want to reduce our headcount and we’ve seen that with all the layoffs across very big tech players, but what does that then look like in your eyes?

Sushila [00:15:51]:
There is definitely a workforce transformation going on that at the moment, where AI being a co pilot was doing simple kind of things for us, like checking reports of producing first drafts of things where you would go through and check it. And what it was doing is it was kind of displacing the apprentice, right, because the apprentice was the person that created the first draft of the slides. Now suddenly, and you’re almost seeing that, you’re seeing that maybe job starters are most impacted because the more skilled people are the people that actually can check the work of the AI. And because it’s still early days, we still need that human being checking the work of the AI. But at the moment we’re more intelligent than the AI. The AI is, you know, a co pilot. Depending on which AI company you believe, you know, around the quarter X number of years, we’re going to have an AI with general intelligence and that, you know, the aim is constantly to improve the intelligence of the AI. And if you believe that is actually possible, then this becomes a workmate that is actually consistently climbing the ladder, growing more skills.

Sushila [00:17:06]:
In fact, Anthropic was talking about how they’re developing AIs that develop their own AIs. And we’re talking about coders, for instance, really getting hit right by AI, because AI is very good at languages, programming languages especially. It’s good at going out and reading legacy code. Some tasks, it’s really kind of optimal around. And so will we always need human beings in the loop checking the results of AI, or will it be another AI checking the results of AI? Ultimately, I believe these organizations are spending millions of dollars because they believe they can save millions of dollars. And so the question would be, right, some of these jobs here looking to replace with automation, they’re going to need domain experts because that’s where the information comes from. You can’t develop an AI healthcare magic without someone that really understands healthcare. And we’re going to need people to be able to test these AI systems, to be able to construct the kill switch to avoid the HAL problem.

Sushila [00:18:18]:
Space Odyssey HAL KILLING the human race so we’ve got a lot of work to do on this, but different kinds of work, and that’s also what we’re seeing. In the workforce is that the skills that we’re needing is really changing, and it’s changing so fast. And you notice some people are so good at adapting. Like, they roll in, you know, their first person wearing the. Whatever it is, right. They’re in with the trend. And there are other people going, you know, what? If we close our eyes, it’ll be gone. It’s just like.

Sushila [00:18:46]:
It’s just a fashion. I don’t think it is. You know, I’d say here to stay. And my recommendation would be for people to use it every day in their life so that they learn how to work with it. Because the way we ask questions, every time you’re using more than one question, it’s spinning cycles, it’s spending tokens, it’s more expensive for the company. If you learn to have more efficient, optimal questions with an AI, then you’re getting better and better. And if you learn how to use AI agents, you’re developing the skills that are very, very much needed. However, I will say the reason why people automate is to optimize and to save money.

Sushila [00:19:28]:
And then you have to ask, how are they saving that money?

KB [00:19:33]:
But isn’t it like they’re sort of. What’s the analogy? Robbing Peter to pay Paul? So, like, whilst they’re saving money, but they’re also investing a lot of money to get the tools in the first place? So then is it a moot point?

Sushila [00:19:44]:
Clearly, they do not believe so. And the logic would go, did we save money by dumping the horse and cart and swapping it for a car? Well, the problem is, if you were driving the horse and cart, you couldn’t keep up with a car. Today, it almost becomes you have no choice because we’ve transformed the way that we do commerce or how we do transport. And so I think we’re really where if you don’t leverage AI, you won’t be able to move at the same speed as someone else. I used to do coding years ago, decades ago. Now when I look at coding using these AI tools, it’s just incredible how quickly you can move. Maybe five, six, seven times faster. So a job that would take somebody five days takes them one day, you know, developing or even developing reports, you name it.

Sushila [00:20:45]:
Like, I see it in the way that I work, I’m taking quarter of the time that I used to. So I’m also getting, you know, paid, therefore, a quarter less for the same work that I’m doing. Right. But if I don’t do it, then Joe Bloggs is going down to Cut me. Because they’re going to be using AI and they’re going to be able to produce that work. And of course, quarter of the time, sometimes humans are their own worst enemies. Right. So we’ve actually developed this scenario where just to stay competitive, I have to use AI, even for me in the work that I do because everybody else is.

Sushila [00:21:23]:
And I can’t move as quickly as they are if I’m not using it. So it’s really kind of ironic.

KB [00:21:30]:
And then so would you say is to conclude as well that perhaps in the it’s more like capital expenditure or capex spending money up front or front loading it. So it looks like, oh, we’re spending lots of money right now, but maybe in a year, two years, that won’t be as expensive because the AI is doing stuff, the agents working on its own. We, you know, displace 10 people. So therefore we’ve saved, you know, a couple of million bucks on salaries. So do you think that perhaps this whole AI conversation and the ROI around it will change? Like, if we talk again in a year about, hey, Sushila, where is people’s heads around the ROI? I’m assuming that’s probably going to change significantly.

Sushila [00:22:12]:
That is so beautifully said. You’re I 100%, you know, we have to make that date 100% feel that is exactly what’s going to happen. It’s always the steepest curve to begin with on investments, you know, because we’re learning, we’re excited. Experimenting, we’re getting only some of it. Right. This is the most expensive time, is that, you know, how many. Even when people move to the cloud, right. How many migrations didn’t make, you know, save them the money that they initially thought they would.

Sushila [00:22:44]:
Right. And we’ve just got better and better at it. And right now I think that’s where we are with AI is we’re learning, we’re investing really heavily in it because we believe it’s the only way we could stay competitive. But I think this conversation will be very different in a year around ROI.

KB [00:23:04]:
Joining me now is Mark Thomas, president at Escoute Consulting, board advisor and ISACA hall of Fame entrant. And today we’re discussing if AI could start governing us. So, Mark, thanks for joining me and welcome.

Mark [00:23:16]:
Thank you very much. I’m glad to be here. In fact, it’s very timely. I’m at the ISACA Annual conference in Las Vegas, and in fact, one of the presentations I did yesterday was about when AI begins governing us, and we’re not governing AI. So it’s a great topic to hit right now.

KB [00:23:32]:
Okay, well then let’s hit on that. I’m very excited to explore what you mean by that. Like is that happening now? When’s it going to happen? What are your thoughts? Tell me everything then.

Mark [00:23:40]:
Yeah, so, so, you know, we have a lot of frameworks and everybody thinks we have the right framework that’s going to, that’s going to save our organization. But I’ll talk frameworks here in just a little bit. But I think what’s happening right now is we’re seeing an introduction of a new taxonomy of risk categories here. There’s decision risk, right? We’re worried right now that decision risk replaces system risk that has to take place, right? We need to start thinking because decisions are being made in some organizations, agentic systems that are being unchecked and with no guardrails at all. So where traditional IT risk says, hey, is the system available? Is it secured, is it accurate? AI risk says, was the decision the system made defensible and was there supposed to be a human in that loop? We’re also seeing some new risks about this thing called drift. And nobody had heard about behavioral drift in a system. It’s a category that didn’t exist before. So AI systems can change after deployment.

Mark [00:24:39]:
We should be retraining them, fine tuning them, do prompt engineering and those kinds of things, even though they have their own learning loops. I’m finding this today. I’m in the process, actually. I’ve just finished phase one of building my own digital twin. And my digital twin, I’m a one person company and my digital twin is operating right now in the background as we speak. It also has a little bit of hallucination sometime and will drift a little bit. But the key thing is for organizations to look at this, something called calibration to make sure it has the right voice and so on. One other area to tell you that I’ll kind of sum up these risk categories is accountability.

Mark [00:25:17]:
Diffusion is now the new system risk. I can have agents work on my behalf. But remember, when we have an AI agent that actually has the capability of making decisions, that’s the same thing as with a human, but we’re doing it with a, with an AI model. So I always ask the question, as a human, the decision that you just made, how do you know that you have the authority to make that decision? And what attributes have to be available in order for you to do that? We have to apply the same thing for our AI agents. So we’re seeing that and then of course, the third party risk. Right, because in digital supply chains that now are growing immensely, right. It’s not all about the third parties. The fourth, fifth, sixth, seventh party that might be in this digital supply chain that you may not have control or visibility over.

Mark [00:26:06]:
So those are some of the new risks we’re seeing. Of course, you know, third party risk has been around for quite some time, but in this case we’re starting to see that have a huge effect. And then of course, finally the, you know, upcoming. Well, it’s in our face now. Compliance risk. Compliance risk. New regulatory compliance rules that I’m sure we’ll start talking about here in just a little bit. So those are some kind of key things that are hitting us in the face right now.

KB [00:26:29]:
Okay, I want to get into some of those. So I want to start with decision risk. As you would know, Mark, now companies are making decisions like a lot faster than like back in the day, we’d sit around a room and have tech risk, business risk, security risk. Now companies to stay competitive have to just make the decision. So maybe they are cutting corners on risk. So would you say because of that, and I get it, if we zoom out, companies need to be competitive, stay ahead of the curve, be top dog. That’s obviously really starting to drive those decision risks. I want to talk a little bit more about that because we haven’t, especially in large enterprises, companies have always been a little bit more reserved in how they’ve made decisions.

KB [00:27:10]:
And now that’s fundamentally changed.

Mark [00:27:12]:
Right. I think about Air Canada, you might be familiar. Not too long back, Air Canada had a chatbot, right? They invented a bereavement fair policy, right? And it didn’t exist. And a Canadian tribunal made that airline pay anyway, they said. And the courts basically said, look, you had an agent, technically an agent, make a decision on your behalf. You are accountable for this. That’s not a cybersecurity incident. That’s not a data breach.

Mark [00:27:37]:
That’s an entirely new category where. So at some point a human might have been removed from that loop. So I think removing humans from the loop is probably one of the biggest issues. But we want to speed things up and oftentimes we’re seeing AI automated controls move faster from humans. So an example of where controls that are human based cause a failure. You might remember this is quite some time ago, something called the Flash crash, where some autonomous investment decision making technologies started a frenzy of selling and trading at $3 trillion was lost instantly. It was moving so fast that humans could not stop it. In that process.

Mark [00:28:21]:
And that’s the whole point, is to make sure that humans have some point in this, but we can still automate a lot of those other decisions. In fact, just the first five weeks of me building my digital twin environment, I had not touched a piece of technology. The first five weeks was just about guardrails. Now, when I asked my digital environment it to do something for me, sometimes I will intentionally put some wording in my. In my prompt that asks it to make a decision, and what it will do will come back and say, that is beyond the rule set you’ve allowed me to do. You have the authority to override that? Would you like to? If you do, I’m going to document that decision so that it’s auditable, so that somebody with the proper authority allowed me to do that. And that’s really kind of the key thing, because the shift happens, right. As soon as you start producing information.

Mark [00:29:10]:
Right. And we have output that a human doesn’t review, or another autonomous agent that reviews it as a check that at some point there should be a human maybe taking a look at those. Does that make sense?

KB [00:29:24]:
Totally. Because I want to ask then about the other category that you mentioned before, Mark. So drift. So talk to me a little bit more about that. And I know you sort of spoke a little bit more about the digital twin, but do you. And the reason why I want to talk about this is because people are starting to perhaps sit back a little bit more, let the machines do the work. Like we said, the human in the loop, there’s a digital twin. Of course it hallucinates.

KB [00:29:47]:
But I want to talk about the drifting side of the risk, perhaps. I just think of, like when you’re on a cruise ship and then you go snorkeling and then the ship leaves without you and you’ve drifted away.

Mark [00:29:56]:
That’s a great analogy. Because currents change. Think about drift. So you may create a model based on a certain social, environmental, political environment. Right. As we all know that those types of cultures, external cultures, can change. The same thing can change here. Just like that, Drift.

Mark [00:30:14]:
When you’re snorkeling and you’re on the other side of the boat, you didn’t realize it. So this behavioral drift, it’s a category that really never existed before because we didn’t have machine learning systems that learn. So they change after deployment, the second it deploys. And we have things called retraining, fine tuning, and those types of things. So even in their own learning loops. And so one of the things that actually I do And I’ve done several times now, even in my environment is what’s called a calibration. And so that calibration is where the system, you and the system interact in a series of questions and answers and exercises. And then you get, as a human are evaluating that outcome and saying, yes, we’re still in line with the original intent of voice style, death, ethical types of decisions that need to be and that human is involved as a part of that drift.

Mark [00:31:04]:
These models could change drastically and cause a lot of issues. Let’s take New York City. And I’m not real, I’m not extremely aware of all the details, but somebody was telling me earlier that there was a chat with bot that New York City had created to help and assist small business owners navigate through the rules, the red tape, the, you know, the fees, all those things to help them navigate through this. But what happened with the tool is there was a part of the tool that was never trained and it said, wait a second, if I really am supposed to be helping these folks, let me give them a couple of hints on how they can go around the rules. So it started giving them advice because it was doing exactly what it was doing, trained to do, but it was never told. Hey, you shouldn’t tell them to break the law or try to figure out how to evade a fine or something. So that’s in a case where, where drift was largely check unchecked by humans and who were continuously trying to calibrate and check the outcomes of that.

KB [00:32:02]:
So yeah, okay, so then on that note, with calibration, for example, like how this is like a, how long’s a bees of string, but just generally speaking, like how often should you be calibrating these systems to make sure it’s still on track, we’re still in sync, still on the same page, because ultimately we don’t want to have get around some of those getting around the law sort of problems.

Mark [00:32:23]:
Unfortunately, today there’s not a table that exists that says if you’re this type of organization, here’s where you calibrate this. Because you think about it, these are for individual models. So your organization may have multiple models. In an AI system, you have a model for accounts payable, receivable, hiring, human resources, strategy development, whatnot, right? And these models have something called a model car, a card with it. And a model card is the descriptor of this model. Think of it as the carfax for the car. Right, the car. Well, carfax in the US is a, is a description of a vehicle with all the details you need to know about it.

Mark [00:32:56]:
So that’s that descriptor, you know, kind of asset car, if you will. And it has something associated with that called an algorithm. Okay. And then that algorithm is this is the set of that the model goes through. But that algorithm has a set of rules called hyper parameters around that. And what they do is they say as you’re making this decision, here are your rules and we can modify and change those. My point being this, if we have any, what I would consider significant changes to inputs to maybe our document canon that we might use the addition of new systems and tools that we may be integrating through. Because you know, AI system is not just one, it’s not Claude and it’s not OpenAI.

Mark [00:33:36]:
It could be several things that are linked in adding new, what we call APIs. In that would be a situation where I would do, I would do a calibration check. So I would say that from, from an organizational standpoint, you know, I’m making this up, right? You could say we’re going to do one for each of these models, monthly, quarterly, so on. And again, depends on the size and complexity of the model. But definitely when an event takes place that triggers a potential cause for drift. I think that there would be a calibration exercise just for that model.

KB [00:34:07]:
So do you envision, and like you said, there’s, there’s probably hundreds or thousands of models in a big enterprise, for example, do you think this then leads into like the governance layer, like moving forward? So it’s like every X amount of. I’m just arbitrary. Maybe for HR models, every quarter we need to do a calibration. Someone physically as a human being needs to do that to make sure on the same page. So Mark’s not getting paid a billion dollar bonus at the end of the year because it’s drifted.

Mark [00:34:35]:
That’s right. And nobody catches it until I’ve left and I’m on a beach somewhere. Right. I totally agree. And I think there should be a timeframe for this. And I think as organizations are moving in to this, they’ll see that, hey, we have automation that will tell us, right? My, my system will come to me and tell me, hey Mark, you’ve made some significant changes. Let’s do a little calibration here to make sure that we still sound like you. Right.

Mark [00:35:00]:
But at a minimum, at a minimum, I do it myself for my trade. Again, my environment is nothing like a large corporation. I require my system to prompt me quarterly to do it. And I like kind of going into the governance piece. Because I will tell you this, so many people love to throw the term governance behind something and they just expect governance to magically appear, right? You know, we’ve got corporate governance, finance governance, IT governance. Now we got AI governance. It’s causing a lot of people to freak out saying, oh my gosh, we need to go redo our whole framework. And I tell everybody, wait, get your governance structures right first.

Mark [00:35:40]:
Because there’s so many different things that you can look at. So, so I think about your governance structure. My analogy here is, is your traffic rules, right? You know, a lot of folks, especially in fast moving IT shops, agile DevOps types of shops are like, ah, governance is just their handcuffs, it holds us down. Well, we have to have rules, right? I’m quite comfortable that we have driving rules, right? So because wherever you are in an organization or in life, there’s always some governing body that’s setting some rules for you, right? So when I drive, right, we know I’m, I feel comfortable, I have speed limits and I feel comfortable to know that hey, the rules are we stop at a stop sign, term user signal light, those types of things. But what happens is I may be on an open highway where I’m doing 100, you know, 100 miles an hour. There may be open autobahn, right? But there’s little risk there. But as soon as I get into a city or a metropolitan area, it goes down to 35 to 25, so, so the speed limit changes. So I equate that to governance structures and organization.

Mark [00:36:39]:
You still have your same traffic rules, right? Corporate governance, IT governance. And we’re kind of zoning in and saying, okay, now we have this piece called AI Governance. AI governance does not replace your requirement to have IT governance, risk governance. All the other pieces, we’re adding another layer inside of that. And I’ve got some thoughts on that here in a couple seconds.

KB [00:37:03]:
Okay, so then maybe just on that note, do you think that some of the governance frameworks are just outdated now? At the, at the minute?

Mark [00:37:11]:
Yeah. So you know, I play them in the framework space a lot. In fact, you know, being a part of ISACA, we have a couple of really significant frameworks. Covid is one of them. We have digital trust Ecosystem framework. But I wouldn’t say they’re outdated, I say some now are incomplete. Cobit NIST. NIST has what’s called the AI RMF.

Mark [00:37:30]:
They have the cybersecurity framework, ISO. Some of the more applicable ones would be ISO 27000 series around security and this new one is called ISO 42001. Right. Just focused specifically on an AI management system. So they still work. Right. Each one of them works in their own areas, but especially like the earlier ones, they were designed for systems to make autonomous decisions. And that’s the thing that’s showing up now that we didn’t have before.

Mark [00:37:57]:
And they’ll evolve these a little bit after deployment. So my suggestion to folks is, hey, use these core frameworks and know how they fit together, but just adjust the speed limits inside of them. Right. Because you, because if we go change an entire governance framework because AI is here, I get it. There are so many significant things and we finally get this new framework, guess what we’re going to be doing in less than three years, doing it again when quantum computing hits us in the face. And it’s, we already know it’s inevitable. So get your core governance structures together first. So I would say that the frameworks would not be the problem.

Mark [00:38:36]:
Right. A framework doesn’t make your decisions. A framework doesn’t replace leadership and accountability. It’s the application of the frameworks. Most organizations are bolting AI onto their governance model that were built, you know, and they’re just calling it done. It’s not a, not a create, forget kind of thing. So that was kind of the whole idea of cobit. Right.

Mark [00:38:58]:
To, to we want to have a governance framework over information and technology and as new things are deployed. That’s my core model. Right. It’s not going to change the fact that a change that has to go into a live production environment doesn’t get approved. Right. That’s governance. You can’t add, remove or modify anything unless some type of authority, authority approves, disapproves and schedules that change. What might be happening now is that’s being delegated from say a change advisory board where 14 people get together every Tuesday and talk about the changes to an authorized AI agent who has human oversight.

Mark [00:39:38]:
That is the difference in how the frameworks and how I believe people should be looking at this framework. So the next generation of governance won’t replace the old framework? No, it’ll layer on top of them. And that’s one of the things, you know, with, with ISACA we have a new AI risk certification that kind of addresses that decision making capability. But it’s not a framework though.

KB [00:39:57]:
Yes. I was just going to ask you on that to probably close with our interview today, to, to use your words, layer on top. So it’s called the AAIR or Advanced AI in Risk. Talk me through that. Like, what does this mean? Does that sort of. How does that change sort of what we’re doing now in the industry? Walk me through it.

Mark [00:40:15]:
This is the latest certification for my soccer. In fact, we just taught the inaugural AAIR course the first two days for this conference. I was the instructor for this. But if you think about. So let’s talk aair. So cyber risk, right? Cyber risk is about protecting assets, right? AIR helps us understand not just that protection thing, but it’s out. It’s about how we assess, how we look at the risk and how we defend our decisions. Where traditional models would say, what can go wrong with the system? AAIR focuses on what could go wrong with the outcome.

Mark [00:40:50]:
And could you defend that outcome to your customer, to your boss, or to a regulator or even court? That’s a fundamentally different discipline right there. So it forces. So the courseware and the content is very rich. It forces life cycle thinking. It is not a course that’s going to make you an AI expert by any means, right? It’s a course that’s going to teach you or support the current skills you have in risk governance and risk management. So you got this life cycle of this AI system, right? It looks at the assurance of the entire life cycle, not just a point in time, right? In the old days, right, we used to say, oh, we do a risk assessment every year. Today, risk assessments are continuous, right? A what we call a SOC2 to report or ISO 27000 on it. SNAPSHOT.

Mark [00:41:40]:
AAIR recognizes that this is continuous. The model in production today is materially different, the one that we approved last year and the one the SURES has to keep up with. So. So you can’t certify a moving target with a. With a static framework. And that’s really kind of the whole idea. So this puts accountability before architecture. Because that accountability is the biggest concern of mine, that we’re going to have these rogue things.

Mark [00:42:05]:
And I, you know, I have a funny picture of. I show in. In my courses. It’s got a picture of the Terminator. Remember the Terminator? And on the bottom it says, have you ever said thank you to your AI system? Just in case this happens, people? Everybody says, yeah, I say thank you. I say thank you. And why do you say thank you? Well, it’s just. It’s a nice thing to do.

Mark [00:42:27]:
But also just in case AI goes crazy, it might remember that I said thank you. But so again, we’re looking at that whole accountability part of that. So controls are moving. Controls are dynamic now, and so are the risks. So imagine right imagine a system where I have a dynamic risk register right and this risk register changes not just every day but as new signals are heard and where are these signals coming from? They’re coming from my agentic systems that are listening for things and they’re looking at current risk and they may trigger me and say hey Mark you had a risk that just four hours ago was out there within four hours it the likelihood of this happening I see as being significant at this point. This is the human decision right to say okay here’s what we do next and that is taking place dynamically but it’s an overload of information so accountability before architecture there is an exam associated with that too but you know the real thing is the rich content which is the latest and greatest stuff on AI is what we talk about in that course so it’s very relevant

KB [00:43:36]:
And there you have it. This is KB on the go. Stay tuned for more.