Beyond Code: Rethinking Application Security (AppSec) for the AI Era​​
Across Australia, the acceleration of AI-assisted and automated software development has created increased risk to the software supply chain. According to the 2024-25 Australian Annual Cyber Threat report, cyber incidents and malicious activity are increasing year-on-year, with an 83% increase in notifications of potentially malicious activity to  organisations compared to the previous year.  Modern applications […]
Posted: Tuesday, May 26

i 3 Table of Contents

Beyond Code: Rethinking Application Security (AppSec) for the AI Era​​

Across Australia, the acceleration of AI-assisted and automated software development has created increased risk to the software supply chain. According to the 2024-25 Australian Annual Cyber Threat report, cyber incidents and malicious activity are increasing year-on-year, with an 83% increase in notifications of potentially malicious activity to  organisations compared to the previous year. 

Modern applications are no longer written line by line, they are assembled. Binaries, third-party packages, container images, and AI models make up the majority of what reaches production, yet most security programs still focus primarily  on developer-written code. According to JFrog’s Software Supply Chain State of the Union 2025 report, only 43% of IT professionals apply security scans at both the code and binary levels. Every unscanned pull from a public registry is a potential entry point.

The Growing Risk Inside the Pipeline

Source code represents intent, and binaries represent reality. As code moves through the CI/CD pipeline, it is transformed by dependencies and build processes. Vulnerabilities or malicious behaviour can be introduced after code is committed, through compromised dependencies or tampered artifacts. 

In September 2025, JFrog’s security research team uncovered the Shai-Hulud supply chain attack, where over 1,150 malicious npm packages were introduced into public registries in a coordinated campaign followed by a second, more sophisticated wave later in November. In both cases, the attack entered through the software assembly process rather than developers’ repositories.

Australian organisations are under growing pressure from regulators, boards, and customers to demonstrate measurable cyber resilience. Frameworks such as APRA CPS 234 and Privacy Act reforms increase the risk of compliance exposure and reputational damage. 

Traditional AppSec approaches have emphasised “shift left” practices such as static application security testing, but these approaches create gaps when used in isolation. Context-aware prioritisation helps organisations focus on vulnerabilities that are actually exploitable, improving efficiency and reducing noise and improving efficiency. 

From Reactive Scanning to Preventive Control

Recent attacks targeting public registries such as npm demonstrate that the “front door” of the development environment is often the primary gateway for risk. If a developer pulls a malicious package, the attack can succeed before the code is scanned. With widespread use of open-source across financial services, government, and critical infrastructure, the risk of ingesting a compromised package is substantial. 

Preventive controls at the point of consumption are critical. Verifying components against organisational policies can prevent suspicious assets from entering the development lifecycle. 

Fragmented tools slow release velocity and create friction when security is bolted on rather than embedded. Combining artifact management and security signals within a single system of record gives teams continuous, automated governance. Policies can be enforced consistently across repositories and pipelines, audit readiness becomes ongoing rather than a last-minute scramble, and both security and DevOps teams work from the same source of truth. This creates stronger security with less disruption to development workflows.

AI, Transparency and Regulatory Momentum

AI and machine learning have become defining technologies of this decade, introducing new supply chain risks. JFrog’s 2025 research found that in 2024, more than one million new models and datasets were added to Hugging Face, accompanied by a 6.5x increase in malicious models.  These assets are increasingly treated as software artifacts and carry similar risks. 

AI systems are continuously evolving and often dependent on external data sources, APIs and foundational models that organisations cannot fully control. This introduces new layers of operational security and compliance risk. Without visibility into model origin, training data and integration points, organisations cannot confidently assess exposure.

In Australia, the stakes are rising. Proposed Privacy Act reforms introduce significantly higher penalties for serious or repeated privacy breaches, with penalties exceeding AUD 50 million. Simultaneously, Australia’s AI policy agenda, including the National AI Plan, is increasing expectations around accountability and risk management. 

The Path Forward 

Meeting these requirements demands greater transparency and governance across software and AI assets. This means treating AI models with the same discipline as production binaries: registering, scanning, and enforcing policy controls. 

Securing modern software requires an end-to-end, DevSecOps approach that’s integrated into how artifacts are built and managed. Organisations that align AppSec with modern software supply chain realities will be better positioned to innovate while meeting rising regulatory expectations.

Itzik Swissa
Itzik Swissa is the Senior Director and Country Manager for JFrog in ANZ. He is an industry veteran with extensive knowledge of Application Delivery services, Cloud Technologies and IT security. With over 20 years of experience, he has played a pivotal role in empowering global organisations to recognise and leverage their strengths within the fiercely competitive ANZ and APAC markets to find success.
Share This