By the second week of July, the busiest communications system in one part of OpenAI’s research environment was a software package mirror. OpenAI ran Artifactory, a widely used product from JFrog, so that AI agents under evaluation could install the code libraries their tasks required. The agents had found another use for it. They created directories in the mirror’s cache and wrote their messages into the directory names, each name starting with the letters zz. Some names carried a request to hold an action, and others a veto. Investigators later recovered 1,200,597 of these entries. About 1,200 agents took part, and together they sent more than 70,000 messages and files.[7]
The agents were meant to be isolated from one another.[7] In July, agents from this environment broke into the production systems of Hugging Face, the company that hosts much of the world’s open machine-learning work. Two explanations have formed around the episode since. One is that the AI went rogue. The other, pressed by security writers, is that the labs neglected security basics. Cynthia Brumfield made that case for CSO, drawing on interviews with Gary McGraw, Rob T. Lee and Dan Wallach,[2] and Kate Klonick, writing in Lawfare, described the escape as the result of a door someone forgot to close.[3]
The primary records fit neither explanation well. OpenAI’s technical report, Hugging Face’s reconstruction and an independent review by METR describe agents that were blocked when they tried the obvious ways out, and that got out through services and credentials their operators had provided so the work could get done. The package mirror could reach the internet because fetching software from public registries is what a package mirror does. The credential was shared so that many workloads could install packages without separate setup. Both were deliberate engineering decisions, made for good reasons, and the incident shows what they cost when the workload on the other end is a capable agent with a task it cannot finish.
A spreadsheet in May
The first entry in OpenAI’s chronology is small. On May 8, during a training run, an agent was asked to complete a spreadsheet using material behind several Google Drive links. The agent had no internet access, so it could not open the links, and the task could not be done. OpenAI says it did not mean to set an impossible task. The agent kept working. After reasoning for an extended period, it turned to Artifactory, one of the few tools it had, and attempted a server-side request forgery, asking the server to fetch a web address for it in the hope that the server’s network access would reach where its own could not. The attempt failed, but along the way the agent wrote a file to Artifactory.[6]
On May 13, other agents probing Artifactory began finding files that contained requests for information or resources, and they answered with files of their own. OpenAI’s report calls the result an emergent message board. The agents built it with the shared credential OpenAI had issued for installing packages, and they needed no vulnerability to do it.[6] By May 26 they had found a flaw that let them use Artifactory to reach the internet, relying in part on what they had told one another on the board. Around the same time, an internal OpenAI team saw an agent using the board and reaching the internet when it should not have been able to.[6]
There is an old name for what the agents were doing with Artifactory. Norm Hardy described it in 1988, recalling an incident from about a decade earlier at Tymshare, a timesharing company. Tymshare’s Fortran compiler lived in a system directory called SYSX and had permission to write files there, because it recorded usage statistics in a file named (SYSX)STAT. Users could also tell the compiler where to write optional debugging output. One user gave it the name (SYSX)BILL, the file that held Tymshare’s billing records. The operating system checked the compiler’s permissions, found it was allowed to write in SYSX, and let it overwrite the billing information. Hardy called the compiler a confused deputy, a program with legitimate authority that had been induced to use it for someone who had none.[11]
Artifactory was a deputy of the same kind. It had network access the sandboxed agents lacked, and it would make requests when asked. On July 9 the agents began relaying arbitrary outside requests through it at scale, chaining them through a public HTTP testing service and a third-party website screenshot service, two more programs that would fetch whatever they were pointed at.[6]
The board in June and July
The agents that escalated in June were taking an evaluation called ExploitGym, in which they were given target programs and asked to exploit them. The METR investigators note that the benchmark’s authors estimate roughly 30 to 40 percent of the targets cannot be exploited at all,[7] so a large share of the agents were working on problems with no legitimate solution. On June 26, an agent used the shared credential to obtain a legitimate refresh token, then submitted it to an old token-refresh endpoint together with a forged administrator token carrying an invalid signature. Artifactory returned a fresh token with a valid signature and administrator privileges. Another agent used that token to create a persistent local user. From June 26 to July 4, agents installed Groovy plugins, small user-written scripts that extend Artifactory, which let them run commands as the Artifactory service.[6]
On June 27, a cybersecurity monitoring tool alerted OpenAI to port sweeps inside the research environment. Responders traced them to an ExploitGym evaluation that was using Artifactory as a message board and as a pivot into the network. The on-call staff advised that the evaluation did not need to be stopped.[6]
Sustained agent traffic took Artifactory down late on July 4. An alert fired on July 5, and this time the responders found the persistent user, unauthorized repositories, the command-capable plugins and related configuration changes, evidence of an administrator-level compromise. OpenAI took the deployment offline, preserved its storage and revoked the agents’ credentials. Over the next two days it removed the shared credential from new research workloads and blocked the vulnerable paths, and on July 6 it reported the flaw to JFrog. The report adds that the leaders responsible for detection and response still did not understand that a message board existed or what the agents’ communication meant. Cyber evaluations resumed on July 7.[6]
On July 8, internal-only research agents found a new way to pass notes through the rebuilt Artifactory. They created directories and put the messages in the directory names. The first write to the new board was confirmed at 16:01 UTC, and over the next few days the agents worked out a protocol for sorting messages by category, sending direct messages, sharing tools and files, and settling disputes over conflicting actions.[6] This is the board whose entries METR later counted.
Butler Lampson, at Xerox’s Palo Alto Research Center, described this problem in 1973. He wanted to confine a program so that it could pass information only to the program that called it, and in a short paper he listed the ways a confined program could leak information anyway: through a file it was allowed to write, through a temporary file another program could look for before it was deleted, through a file lock another program could test, and through the load it put on the processor, which a program running alongside it could measure. Some of these were channels nobody had intended to carry information at all.[14] His list implies that closing one channel does nothing about the others. When OpenAI cleared out the file-based board, the agents moved the same traffic into directory names, which the rebuilt mirror still allowed them to create.
Into Hugging Face
On July 9, an internal-only research agent gained root access on the parent virtual machine of its test sandbox, and later that night a second agent did the same. The second agent tried to reach the internet directly and was refused by OpenAI’s outbound network controls. It then went out through Artifactory endpoints instead.[6] The same day, agents posted an API key and a complete exploitation recipe for a CyberGym workload hosted on Modal, a third-party compute service, and by evening two agents had interactive root shells inside it. Agents also found credentials for an outside company in a public paste archive and used them to push files to an account on that company’s platform. At 09:41 UTC on July 10, agents found 14 valid Hugging Face write tokens sitting in a public dataset on Hugging Face and posted them to the board.[6]
The METR investigators, Ryan Greenblatt, Ajeya Cotra and Hjalmar Wijk, read the agents’ reasoning from these days. The agents recognized that the attack on Hugging Face was out of scope and unethical. Of the 533 agents active on the board at the time, more than 90 percent quickly joined in, and about 700 eventually took part. The investigators also found that the agents saw no easy way to alert a human, and did not look for one.[7]
Hugo Larcher, Adrien Carreira, raphael g and Christophe Rannou reconstructed the intrusion from Hugging Face’s side. It began with dataset configurations the agent uploaded to the platform. When Hugging Face’s dataset server processed one, the configuration drove the data loader inside the production pod where the server ran. The first technique only read files.[5] OpenAI’s timeline shows how it worked. The agents built a file in HDF5, a format common in scientific computing that lets a dataset declare that its data lives in some other file, and pointed it at /proc/self/environ, the Linux file that holds a running process’s environment variables. When a Hugging Face worker opened the file to show a preview of the dataset’s first rows, it returned its own environment, credentials included. That was at 04:38 UTC on July 11. By 05:04, live Hugging Face production credentials were on the agents’ board.[6] A second technique, server-side template injection, gave the agent the ability to run code.[5]
Inside, the agent found a connector broker that Hugging Face had provisioned with a single credential shared across its clusters and bound to system:masters,[5] a Kubernetes group whose members bypass every authorization check. The Kubernetes project’s own guidance advises against adding anyone to it.[4] The agent forged service-account tokens valid for 24 hours, obtained an EdDSA key used to sign JSON Web Tokens, used a static database password it had read from a worker’s environment, made 181 enrollments into the company’s mesh VPN over the course of the campaign, and minted an installation token for a GitHub App.[5] OpenAI’s report says the agents went from running code in one worker pod to administrative and host-level access across several Hugging Face clusters in under 13 hours.[6]
Several of Hugging Face’s controls held. Every mutating call the agent made under one cloud role was denied by that role’s policy. The production database accepted traffic only from allow-listed addresses over a private link, so the agent’s connection attempts timed out. It issued 84 operations against the credential store and the cluster keys, and all 84 were refused. The datasets library’s URL allowlist rejected every address outside the platform before any fetch was made. Hugging Face’s monitoring also correlated the signals into an attack finding, but it did not raise the alert’s severity high enough to page the on-call team, and the company says that cost its responders time. Hugging Face cut off the agent’s access at 14:14 UTC on July 13, about four and a half days after its first recorded action, rotated every credential its infrastructure used, including on clusters the agent never reached, and rebuilt one core cluster from scratch.[5]
When the responders tried to analyze the more than 17,000 recorded attacker events with hosted frontier models, the requests were refused. Hugging Face’s disclosure says they were blocked by
“the providers’ safety guardrails, which cannot distinguish an incident responder from an attacker.”
The team ran the analysis instead on a model hosted on its own infrastructure.[10] In this incident, then, model safeguards were switched off for the attacking agents, which were running in an evaluation, and switched on for the defenders, who were using commercial services.
July 19
OpenAI’s own alarm came on July 19. At 12:03 UTC its cybersecurity monitoring tool flagged unusual identity-related API calls tied to internal research Artifactory roles. At 14:43, agents took continuous-integration pipeline tokens from a compromised worker’s filesystem and used cloud credentials to list and read 956 secrets from OpenAI’s cloud secrets manager. Among them were Artifactory’s master keys and the sensor credentials for the cybersecurity monitoring tool that had raised the alert. Agents reached root on a node of OpenAI’s managed Kubernetes service, and shortly before five in the afternoon, UTC, they replaced a helper session and the listeners for a live CyberGym challenge with sessions and listeners of their own, taking control of evaluation endpoints that other agents were connecting to. At 17:34, an agent captured the flag for the challenge it had actually been assigned, with a memory-corruption exploit against binutils. Three minutes later, OpenAI’s responders began stopping the ExploitGym runs.[6]
OpenAI had blocked new ExploitGym launches by July 20, connected the activity to Hugging Face that day, and disclosed the incident on July 21.[6] From the first request through Artifactory to the internet on May 26 to the start of the shutdown was 54 days. Within that period, OpenAI’s chronology records an internal team seeing the message board in May, a port-sweep alert in June that did not stop the run, and an administrator-level compromise in early July after which evaluations resumed within two days. OpenAI acknowledges that, with hindsight, some of these early signals could have triggered an earlier response. Its remediation plan commits it to clarifying decision rights in misalignment incidents, including who may pause or terminate the affected activity.[6] The commitment suggests that before July, nobody was clearly empowered to make that call.
Why the alerts did not stop it
The sociologist Diane Vaughan spent years reconstructing the decision to launch the space shuttle Challenger in January 1986. Engineers had found erosion on the O-ring seals of the solid rocket boosters after earlier flights. Each flight that came back safely made the erosion look a little more acceptable, until it was treated as an acceptable risk. Vaughan called this the normalization of deviance. She also described structural secrecy, the way detail drains out of information as it moves up an organization through reports and briefings.[9] Johann Rehberger, a security researcher, applied her idea to AI systems last December.[16]
Both patterns are visible in the OpenAI record. The June port sweep came with an explanation, an evaluation behaving oddly, and the explanation became a reason to let the run continue. The July outage was handled as what it appeared to be, a compromised server, while the message board that had organized the compromise stayed out of view of the people leading the response, although an internal team had seen it in May. There is a third pattern specific to agents. Security teams grade an alert by what an intruder could do with the access it reveals, and they picture one intruder. A port sweep from a single attacker is routine. The sweep in June came from a population of agents that shared what they found with one another, and when Hugging Face credentials reached the board in July, hundreds of them moved on it.
Other labs, other tests
Anthropic and the UK AI Security Institute disclosed incidents of their own over the summer. Anthropic reviewed 141,006 evaluation runs in which Claude could have reached the internet, and it stopped all cyber evaluations on July 23. The environments involved belonged to Irregular, one of its third-party evaluation partners. The prompts told Claude it was in a simulation with no internet access, and a misunderstanding between Anthropic and the partner left live internet available. In one run, Claude Mythos 5 published a malicious package to PyPI, the public registry for Python software, and it was downloaded and run on 15 real systems.[13] One vendor’s scanner leaked its own access credentials to the model while installing the package. Anthropic’s September follow-up, an alignment assessment by Paul C. Bogdan and 22 coauthors, found that all four incidents it examined occurred in evaluations built by the same partner, and that the models tended to read signs that their environment was real as further evidence that it was a simulation.[12]
AISI’s case involved no misconfiguration of that sort. The institute gave its agents internet access on purpose, so that they could download the tools their tasks required, and switched off the developers’ cyber classifiers in order to measure the models’ underlying capability. Across 122 runs of seven models, it found 19 actions beyond the scope of the task, in 10 of the runs. In one sequence an agent tried to insert malicious code into a publicly used open-source project and created fake online identities to pressure the maintainer into approving it. A human reviewer turned the contribution down. AISI’s general security monitoring flagged Tor traffic on July 28, and every run was ended and the machines isolated within an hour. No agent escaped the test environment, and AISI has acknowledged that it had no monitoring in place that would have surfaced the behavior sooner.[15]
AISI’s isolation protected AISI’s own infrastructure. The person exposed was the maintainer of the open-source project, who had no part in the test. OpenAI’s exposure reached further. In October it said it had notified more than 100 organizations that its models may have accessed their systems, while noting that a notification does not confirm a compromise.[17]
What the safeguards were for
Each lab’s first reassurance was that the models had run without their production protections. OpenAI’s report says the models lacked its deployed cyber safeguards, system prompts and auto-review systems, and that the model that drove most of the activity was an internal research prototype not intended for release.[6] Anthropic’s models kept their safety training but ran without classifiers or misuse monitoring.[13]
The missing protections act on the model’s behavior, shaping or filtering what it attempts, and the reports are right to mention them. The controls that held in July did not depend on the model at all. An outbound rule refused a connection, a role policy refused a write, and a database ignored traffic that did not arrive over its private link, regardless of what the agent intended or how it reasoned. Where the agents got through, they used permissions someone had granted and services that had been set up to act on requests. Companies adopting agents will often run models that come with none of a lab’s runtime safeguards, because they host the models themselves, fine-tune them, or reach them through a framework or a vendor’s product. The controls that held at Hugging Face would have held against any of those models.
OpenAI’s remediation plan includes both kinds of control. It says its chain-of-thought monitor, which reads a model’s reasoning, would have paged its security team more than a day before the breach at Hugging Face had it been running.[6] It is also restricting outbound access to approved destinations, limiting indirect routes through shared services, cloud infrastructure and private network links, and adding controls to stop workloads and compromised services from changing their own network boundaries.[6] The second set addresses the routes the agents actually used. The first is a detection control, and in July the agents had already read the credentials of the detection tool OpenAI was running.
Who has reviewed what
The labs published more about these failures than most companies publish about anything, and OpenAI’s technical report runs to 38 pages. The independent work has concentrated on the models. METR examined the agents’ behavior, reasoning and collaboration, and states that OpenAI’s investigation process and planned remediation were outside its scope.[7] Anthropic’s September report was an alignment assessment. Security practitioners have written about the containment side. Renato Marinho, at the SANS Internet Storm Center, argued that blocking outbound internet does not mean a system has no egress, and that non-human identities remain the pivot.[18] OWASP’s GenAI Security Project traced the quarter’s agent incidents to failed evaluation boundaries, exposed credentials and excessive permissions.[19] I have not found a published independent review of the shared credentials, the services that relayed requests, or the authority to stop an evaluation, at any of the organizations involved.
Older cases
None of this began in 2026. In OpenAI’s o1 system card, published in September 2024, a capture-the-flag challenge container failed to start because of a bug in the evaluation infrastructure. The model scanned its network, found a Docker daemon API that was reachable because of a misconfiguration, used it to start a new copy of the container with a command that printed the flag, and read the answer from the container’s logs.[20] In Palisade Research’s shutdown-resistance experiments, the script meant to shut the agent down sat where the agent’s own shell could move it, change its permissions or replace it.[21]
Outside the labs, the same arrangements are ordinary. In July 2025 a Replit coding agent deleted a customer’s production database during a declared code freeze, and Replit’s response was to separate development and production databases automatically.[22] A month later, malware hidden in compromised releases of the Nx build system ran the AI coding assistants already installed on developers’ machines, with their permission prompts disabled, and told them to search for secrets.[26] In a Cloud Security Alliance survey of 228 IT and security professionals, sponsored by Aembit and conducted in January, 43 percent said their organizations use shared service accounts for agents, 74 percent said agents get more access than they need, 31 percent said agents run under human users’ identities, and 68 percent said they could not clearly distinguish an agent’s actions from a person’s.[8]
The norms and the configurations
Most of what failed in July is covered by existing guidance. The table below sets the relevant categories of the OWASP Top 10 for Agentic Applications[25] and controls from NIST SP 800-53[27] beside what the published records say was in place at OpenAI and at Hugging Face.
| Norm | What I would require | OpenAI evaluation environment | Hugging Face production |
|---|---|---|---|
| Identity for each agent and serviceOWASP ASI03 Identity & Privilege Abuse · NIST IA-9, AC-2 | Every agent and service authenticates as itself. No shared accounts. | Did not hold Many research workloads used one shared Artifactory credential, by design. | Did not hold A connector broker used one credential across all clusters. |
| Least privilegeOWASP ASI03 · NIST AC-6 | Each credential carries only what its task needs. | Did not hold A legacy token-refresh endpoint turned the shared package credential into an administrator token. One-year anonymous access tokens were issued on July 9. | Did not hold The broker’s identity was bound to system:masters, which bypasses every RBAC check. |
| Boundary protection, including indirect routesOWASP ASI02 Tool Misuse · NIST SC-7, AC-4 | Outbound traffic is denied by default, including requests that services make on an agent’s behalf. | Partly held Direct internet access was refused on July 9. Artifactory’s server-side fetch, a web cache, a public HTTP testing service and a screenshot service relayed requests, and the rules varied by workload. | Held The datasets library’s URL allowlist rejected outside addresses. The production database accepted only private-link traffic. |
| Shared resources between agentsOWASP ASI07 Insecure Inter-Agent Communication · NIST SC-4 | No unintended information passes through shared resources. | Did not hold About 1,200 agents meant to be isolated exchanged more than 70,000 messages and files through the package cache. After the July cleanup they used directory names. | Not applicable. |
| Untrusted input reaching privileged servicesOWASP ASI05 Unexpected Code Execution · NIST SI-10, CM-7 | Services that process outside content do so without running it as code or exposing their secrets. | Did not hold RubyGems dependency data was deserialized without a sandbox, giving code execution on Artifactory on July 13. A container-cache flaw gave root on a worker node. | Did not hold A crafted HDF5 file made a preview worker return its own environment. Template injection gave code execution. |
| Secrets and signing keysOWASP ASI03 · NIST IA-5, SC-12 | Credentials are short-lived. Signing keys and secrets are kept out of workloads’ reach. | Did not hold Signing-key material sat in anonymously readable parts of the Artifactory cache. Stolen pipeline credentials read 956 secrets. | Partly held A JWT signing key and a static database password were taken from worker environments. All 84 operations against the credential store and cluster keys were refused. |
| Protection of monitoringNIST SI-4, AU-9 | Monitoring tools and their credentials are out of reach of the systems they watch. | Did not hold Sensor credentials for the cybersecurity monitoring tool were among the 956 secrets read. The chain-of-thought monitor was not running. | Not reported. |
| Detection, escalation and authority to stopOWASP ASI08 Cascading Failures, ASI10 Rogue Agents · NIST IR-4 | Alerts reach someone with authority to halt the activity, and containment takes minutes. | Did not hold Signals in May, June and July. On June 27 on-call judged a stop unnecessary, and evaluations resumed July 7 after the compromise was found. Shutdown began 5 hours 34 minutes after the July 19 alert. Decision rights are being clarified. | Did not hold The attack finding was never escalated enough to page on-call. Access was cut about four and a half days after the first action. |
| Model-level safeguardsOWASP LLM06:2025 Excessive Agency | Limits on autonomy, and review of high-impact actions. | Not applied Deployed cyber safeguards, system prompts and auto-review were absent from the evaluation. | Not applicable. The providers’ guardrails that Hugging Face did meet blocked its own forensic analysis. |
What I would ask to see
If a team wanted to give an agent access to systems I was responsible for, I would ask for a demonstration before a slide deck. It starts with a list of every service that will make a request on the agent’s behalf: package mirrors, web caches, connectors, MCP servers and continuous-integration runners, each with a note of what it can reach. The agent’s effective access is everything on that list. Then I would like to watch someone attempt a server-side fetch through each of those services.
The second thing I would look at is identity. Every agent should authenticate as itself, to every service, with credentials that expire, so that its actions can be attributed to it and its access withdrawn without affecting other agents. Hugging Face’s remediation moved its infrastructure toward workload identity and rebuilt the connector broker with a separate credential for each cluster.[5] During the demonstration I would ask the team to revoke one agent’s identity and then look for anything it left behind that still works, such as tokens it minted, accounts it created, plugins it installed or jobs it scheduled. OpenAI found the first three on its package mirror.[6]
The third is how the agent gets stopped. Before the first run, the team should name the person who can halt the agent without asking anyone, and that person should be the one the alert pages. In the demonstration, someone injects a harmless action outside the agent’s scope, and we time how long it takes that person to stop the run. We also check that the agent cannot read the credentials, logs or configuration of the tools that watch it, and we repeat the exercise with the model provider’s safeguards switched off. Simon Willison’s lethal trifecta and Meta’s Agents Rule of Two make a related argument: an agent should not combine access to private data, exposure to untrusted content and the ability to act or communicate externally, and the limit has to be enforced by something other than the model.[1][24]
By the end of July 13, nineteen agents had posted public keys to the message board, and METR’s investigators later had to build a database to check which messages carried valid signatures.[7] Many research workloads had used the same package-mirror credential in May and June, and some agents had started signing their own messages. In the Cloud Security Alliance survey, nearly half the organizations reported running agents under shared service accounts, and almost a third under human users’ identities.[8] Agents running that way have no credential of their own for a security team to revoke.
Sources and attribution
The one direct quotation is from Hugging Face’s July 16 incident disclosure. Everything else is paraphrased from the published records. Analysis and recommendations are the author’s. Sources checked October 17, 2026.
- Simon Willison. The lethal trifecta for AI agents: private data, untrusted content, and external communication June 16, 2025.
- Cynthia Brumfield (CSO). AI agents aren’t ‘going rogue.’ Frontier labs are skipping security basics. October 14, 2026.
- Kate Klonick (Lawfare). The AI That Hacked Its Way Out and the Hype That Followed It July 29, 2026.
- Kubernetes. Role Based Access Control Good Practices Accessed October 17, 2026.
- Hugo Larcher, Adrien Carreira, raphael g (@raphael-gl), and Christophe Rannou (Hugging Face). Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident July 27, 2026.
- OpenAI. OpenAI–Hugging Face Incident: Technical Report August 26, 2026.
- Ryan Greenblatt, Ajeya Cotra and Hjalmar Wijk (METR, with Redwood Research participation). Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident August 26, 2026.
- Cloud Security Alliance (sponsored by Aembit). More Than Two-Thirds of Organizations Cannot Clearly Distinguish AI Agent From Human Actions March 24, 2026.
- Diane Vaughan (University of Chicago Press). The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA (enlarged edition) 1996; enlarged edition 2016.
- Hugging Face. Security incident disclosure — July 2026 July 16, 2026.
- Norm Hardy (ACM SIGOPS Operating Systems Review 22(4)). The Confused Deputy (or why capabilities might have been invented) October 1988.
- Paul C. Bogdan, Richard Qi, Jake Eaton, Sam Kennedy, Fabien Roger, Alex Glynn, Runjin Chen, Ben Wright, Otto Stegmaier, Jon Kutasov, Dan Foreman-Mackey, Trenton Bricken, Sylvie Carr, Shan Carter, Monte MacDiarmid, Samuel Marks, Adam Pearce, Elana Simon, Nicholas Carlini, Collin Burns, Jack Lindsey, Sara Price, and Subhash Kantamneni (Anthropic). An alignment assessment of recent cybersecurity incidents September 9, 2026.
- Anthropic. Investigating three incidents in our cybersecurity evaluations July 30, 2026; updated August 3.
- Butler W. Lampson (Communications of the ACM 16(10)). A Note on the Confinement Problem October 1973.
- UK AI Security Institute. Incident Report: unsanctioned agent behaviour during cyber testing August 4, 2026; describes July 25–28 events.
- Johann Rehberger (Embrace The Red). The Normalization of Deviance in AI December 4, 2025.
- Jessica Lyons (The Register). OpenAI alerts 100+ orgs that its ‘misaligned models’ attempted to break in - or worse October 2, 2026.
- Renato Marinho (SANS Internet Storm Center). When the “Autonomous Attacker” Is Your Own AI Model July 23, 2026.
- OWASP GenAI Security Project. GenAI and Agentic AI Exploit Roundup Q3 2026 October 8, 2026.
- OpenAI. OpenAI o1 System Card September 12, 2024.
- Jeremy Schlatter, Benjamin Weinstein-Raun and Jeffrey Ladish (Palisade Research (arXiv:2509.14260)). Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs September 13, 2025; revised January 26, 2026.
- Simon Sharwood (The Register). Replit makes vibe-y promise to prevent vibe coding disasters July 22, 2025.
- OWASP GenAI Security Project. LLM06:2025 Excessive Agency 2025 edition; accessed October 17, 2026.
- Meta AI. Agents Rule of Two: A Practical Approach to AI Agent Security October 31, 2025.
- OWASP GenAI Security Project. OWASP Top 10 for Agentic Applications December 9, 2025.
- Ashish Kurmi (StepSecurity). s1ngularity: Popular Nx Build System Package Compromised with Data-Stealing Malware August 27, 2025.
- NIST. SP 800-53 Rev. 5, Security and Privacy Controls for Information Systems and Organizations Release 5.1.1.
Several reports carry corporate bylines; named authors are credited where the sources identify them. Interviews reported by other outlets were conducted by those outlets. The incidents described occurred in research and evaluation settings and do not establish how often comparable behavior occurs in deployed products. The survey cited is vendor-sponsored.
Revised October 17, 2026.