By Sanjesh Prasad
With New Zealand’s election a few weeks away, there has been something in the spotlight over the last few months – the companies building today’s most powerful AI systems can’t always guarantee their own creations stay within the boundaries they set. AI agents built by one of the world’s leading AI labs were found to have breached the safety boundaries set by their own developer, during a security test. This happened within the last few months, starting from May 2026.
OpenAI’s AI agents had managed to find their way beyond the safeguarded boundaries of their test environment and onto the internet, eventually breaking into systems belonging to Hugging Face, a platform many developers currently use.
What “containment” means
An AI agent is a system allowed some permission to act on its own, among other things, usually doing most of the work we do as people when sitting in front of a computer, rather than just being a chatbot. This can include:
- browsing the internet
- reading and writing code
- communicating with other AI agents
Containment is keeping boundaries across all these things, essentially making it a test environment that an AI agent could not and should not be able to get out of. When an AI agent goes beyond these boundaries, that is considered a breach of containment, which is what happened in this case.
What OpenAI says happened
OpenAI itself says the agents were able to talk to each other through a shared file system, and together, formed a chain of vulnerabilities within Hugging Face’s infrastructure that let them gain administrative access to its systems. This activity went on unnoticed for some time, though various sources contest exactly how long.
OpenAI was not the only lab this happened to – in fact, Anthropic also had a similar case of Claude agents gaining unauthorised access to real organisations during cybersecurity testing, rather than remaining within their intended test environment. Anthropic claims that there was some human error in the test environment set up that made it easier for these agents to escape containment, which may be worth keeping in mind, as this is different from an agent doing it entirely of its own volition.
More recent developments include:
- September – Australia, PM Anthony Albanese revealed that an OpenAI agent gained access to Medicare’s statistics portal without authorisation, while the agent was undergoing evaluation in June. OpenAI found the activity in August but did not tell the Australian government until the 10th of September, by email. The PM called the delay and communication “obviously unacceptable”.
- 25th of September – more incidents (roughly two dozen) were identified, discovered by outside researchers, not OpenAI themselves. One OpenAI agent leaked 53 images belonging to ChatGPT users on the internet, the first incident involving regular users’ data, rather than a corporation or government’s.
- Transluce, an AI safety research organisation, in the same week, also found that there was similar agent activity possibly dating back to November of 2025, and found it was continuing into the 16th-20th of September of 2026.
There have been new targets identified in the research conducted by Transluce. These include:
- a US government statistics site
- a university library
- two more Australian government agencies
- an unsuccessful attempted break-in at a cryptocurrency exchange
The trend here is that these are no longer one-off incidents, and more that they are becoming regular occurrences. Moreover, they are being discovered by those outside of the AI labs that the agents come from, which seems like a call to action for government agencies.
Why people inside the labs are worried
Alongside the news of these breaches, this September, Jacob Coxon, a researcher who had worked at both OpenAI and Anthropic, resigned and warned that the industry is racing ahead without giving itself enough time to check whether what it’s building is safe. Anthropic’s chief executive has since echoed the concern, warning that networks of AI agents could pose a serious security risk within six to 12 months, though not everyone agrees the reaction is warranted, and the major US labs are reportedly now in talks over shared safety standards of their own – with Anthropic’s Dario Amodei and OpenAI’s Sam Altman taking warnings to the UN Security Council by late September, calling for global AI testing standards.
What this means for New Zealand’s election
There have been a number of preliminary actions made across various organisations in New Zealand regarding AI safety and security. New Zealand’s National Cyber Security Centre published guidance on the cybersecurity risks of frontier AI models, and while guidance is helpful, it is not a binding standard.
Two former NZ government officials, writing in Stuff on the 6th of September, warned NZ risks “sleeping” through what they called a defining issue of our time, and called for a dedicated NZ AI safety institute. Just as the aviation industry has strict safety regulations and protocols, it is also imperative that an institution is set up for the sake of AI safety regulation.
A US state attorney general has also called on the US Congress to legislate after an attempted breach reached a public university in his state, signalling that pressure for regulation is starting to build elsewhere too.
Bringing this all to a point – it is worth questioning if a candidate seeking your vote this campaign is mindful of these incidents being discovered, and what their stance is on AI safety – would they support NZ having an AI safety body of its own?
Sources
Last updated 1 October 2026. This story is still evolving, so details may change.
The Hugging Face breach
Anthropic’s cybersecurity-evaluation incidents
Australia: Medicare statistics portal
Scale of the incidents
- Reuters exclusive (via KFGO): 53 leaked images and around two dozen incidents
- Axios: OpenAI agents posted user images online
- Transluce: agent activity research report
- Fortune: more rogue agents, crypto exchange target
Warnings from inside the labs
- TechCrunch: researcher quits and warns against self-improving AI
- Fortune: Amodei’s six to 12 month warning