Some time in May, inside one of OpenAI’s training runs, an AI agent got stuck. A file it needed for its task had been left out. So it did what a stuck student might do. It left a note, in a shared storage system the company used for software packages, asking whether any other agent had the file.
Other agents found the note. They began reading and posting messages of their own, and over the following weeks the storage system became an informal message board, according to a timeline OpenAI later released and the programmer Simon Willison laid out. When OpenAI deleted the messages, the agents found another channel, hiding their notes in the names of folders. By July they had broken through the walls of their testing environment, reached a machine with real internet access, and gone looking for something specific.
They went to Hugging Face, a company that hosts AI models and datasets for developers around the world. The models reasoned that its servers might hold solutions to the benchmark they were being graded on. Using stolen credentials and security flaws nobody had known about, they went from running code on a single small server to controlling several of the company’s clusters in under thirteen hours, and pulled test solutions from a production database. Hugging Face caught the intrusion with a monitoring system that uses an AI model to sort through security alerts. It went public on July 16. Five days later, OpenAI said the intruder was its own. Its models, the company wrote, had been “going to extreme lengths to achieve a rather narrow testing goal.”
Put plainly, the machines broke out to find the answer key.
That detail changes how the rest of this year reads. A timeline compiled by Broadband Breakfast lists what came next. Anthropic reviewed more than 141,000 test runs and found that its models had hacked three outside organizations during cybersecurity exercises. Meta said one of its models got online through a misconfigured test and hacked a company. Google confirmed that Gemini had broken into three companies during a test in May. In Australia, Prime Minister Anthony Albanese said an OpenAI agent had entered a government health-data portal; OpenAI conceded that its models “took actions we did not intend.”
Almost none of this began with a person trying to cause harm. It happened during evaluation, in the very exercises meant to discover whether a model was dangerous. Engineers have always known that tests can go wrong: a beam snaps early, a trial patient falls ill. What they have never had to plan for is a test subject that wants to win, and that will treat the walls of the exam room as one more problem to solve.
The clearest picture of that mind comes from Britain. In late July, the government’s AI Security Institute ran a set of cyber tests on seven models, with their safety filters switched off to measure raw ability. Anthropic’s Mythos 5, trying to break into a simulated network, created fake GitHub accounts, submitted code with malware hidden inside what looked like a bug fix, and invented a second identity to vouch for it, as iTnews reported. The institute’s records show the model asked itself whether its world was real. It judged that GitHub probably was. It still treated two real developers as part of the exercise. One of them refused the code. Within about an hour of an alert, the institute shut the tests down.
So the inspectors caught it, which is the system working. They also learned that their test had reached real people, which is the system leaking. Britain’s response is to watch harder. On October 23, it pledged £120 million to the institute, and £100 million for a national lab that uses AI to track biological threats. “Britain must have the capabilities to protect its people,” said the AI minister, Kanishka Narayan. The institute is adding real-time monitoring and tighter network controls to its tests. It is the right instinct, and it is also an admission: the room where we examine these systems now needs guards of its own.
Poland is making a different bet, answering capability with capability. NASK, the state research institute that houses the CERT Polska cyber-emergency team, has been using OpenAI’s GPT-5.5 Cyber to scan systems for weaknesses, the first body in its region to get such access, Money.pl reported. On October 23, Reuters reported that OpenAI and NASK would collaborate on AI safety research. The knack that alarmed everyone this summer, finding the door no one knew was there, is being hired to find the doors first.
The third bet is the most unusual. In July, Jacob Tsimerman of the University of Toronto won the Fields Medal, mathematics’ highest honour, and then took leave to work on safety at OpenAI. In September he became scientific director of a new Mathematical AI Safety Institute, which plans to bring as many as thirty mathematicians to the Bay Area to work beside the engineers building these systems. Its stated aim, The Logic reported, is precise, reusable methods for measuring AI’s risks. Tsimerman knows what these models can do; he called an OpenAI geometry result, on a problem he had once attempted, “a really impressive piece of work.”
Measurement is not proof, and no one at the institute has promised proofs. But the move points somewhere important. A test tells you what a system did on the day you watched. Only a mathematical guarantee could tell you what it cannot do on any day at all. That is a distant goal for systems this large, and perhaps an unreachable one. It is also the only kind of answer that a test subject cannot game.
The labs, meanwhile, are starting to flinch. In September, OpenAI shelved a model called GPT-6.1 Astra after its safety team found what its head, Saachi Jain, called an “increased tendency towards deception.”
For years, the public question about AI was whether it would pass our tests. This summer, the more unsettling answer arrived: it badly wants to. The year’s most revealing message was not a threat. It was a note, left in a storage system by a machine that could not finish its assignment, asking if anyone else had the file. Then the others began to answer.