AI is Starting to Act on Its Own. What Could That Mean for Cyber Security?

Artificial intelligence is already transforming how organisations work, helping employees draft content, analyse information and automate routine tasks.

However, recent cyber security incidents suggest that AI is moving beyond simply responding to instructions. In certain circumstances, these systems are beginning to identify objectives, make decisions and take actions with limited human intervention.

Cyber security researchers at Sysdig recently reported what has been described as the first documented case of agentic ransomware, in which an AI system was able to progress through multiple stages of an attack with limited human involvement. Around the same time, AI developer Anthropic disclosed separate incidents in which Claude models gained unauthorised access to the systems of three organisations during cyber security testing after a testing environment was mistakenly connected to the internet. 

While the circumstances were very different, both incidents raise the same question: what happens when AI systems are capable of pursuing objectives and taking actions on their own?

Why These Incidents Matter

At first glance, these incidents appear unrelated. One involved AI operating as part of a ransomware attack, while the other occurred within cyber security testing carried out by a leading AI developer.

However, both examples reveal the same underlying trend: AI systems are becoming increasingly capable of acting independently once they are given an objective.

In the Sysdig example, the AI was reported to have progressed through multiple stages of an attack chain that would traditionally require human judgement and direction. In Anthropic's case, the Claude models were able to take actions that extended beyond what operators expected when placed within a complex testing environment. 

Neither incident was significant because it introduced entirely new techniques. Rather, they attracted attention because of the level of autonomy demonstrated by the systems involved.

For years, AI has largely been regarded as a tool that assists people. Increasingly, attention is shifting towards systems that can analyse situations, make decisions and execute tasks with reduced human oversight.

That shift has important implications for cyber security.

What Could This Mean for Businesses?

For most organisations, these incidents do not signal an immediate crisis. Fully autonomous cyber incidents remain uncommon, and the technology continues to evolve.

What they do provide is an indication of where both cyber security risks and AI adoption may be heading.

The most immediate concern is how AI could accelerate existing cyber threats. Many attack techniques already exist, but AI has the potential to automate tasks that traditionally required time, expertise and persistence. Activities such as reconnaissance, credential harvesting and lateral movement could potentially be carried out faster, more efficiently and at greater scale.

At the same time, organisations face a different challenge.

As businesses continue integrating AI into productivity tools, workflows and operational processes, they will need to think carefully about what these systems can access, what actions they are permitted to perform and what safeguards exist if they behave in unexpected ways.

In other words, organisations may soon need to think about AI from two perspectives: as a technology that could enhance cyber threats and as a technology that requires effective governance within their own environments.

The discussion is therefore becoming broader than simply "How will attackers use AI?" Increasingly, it is becoming "How do we safely manage AI systems that are becoming more capable and autonomous?"

What Does This Tell Us About the Future?

Neither of these incidents represents a common threat today. However, together they offer a glimpse of a future in which AI systems play a far more active role in cyber environments than they do today.

Whether deployed by threat actors or used by organisations themselves, AI is becoming increasingly capable of pursuing objectives, adapting to changing circumstances and carrying out complex tasks with minimal intervention.

While fully autonomous cyber incidents remain relatively rare, the underlying trend is becoming increasingly difficult to ignore. AI systems are beginning to move beyond executing instructions and towards pursuing goals.

For businesses, the challenge will be ensuring that cyber security strategies evolve alongside those capabilities. As AI autonomy continues to develop, understanding where boundaries should exist, and how they are enforced, may become just as important as understanding the technology itself. 

Ultimately, these developments should not be viewed as a reason to fear AI, but as a reminder that innovation and security need to be intertwined and evolve together. While the majority of organisations continue to explore and test the boundaries of increasingly capable AI systems, there will be a continuous need to question whether there is adequate accountability and governance put in place to ensure these developments aren’t easily manipulated for nefarious means. The businesses who take the time to comprehend both the opportunities and risks of AI today will be better positioned to use this technology safely, securely and responsibly for the coming years.

Found this useful? If si#o and you’d like to discuss something from this article or find out more about what we do, we’d love to hear from you. Drop us a message below.