The Story of an AI Told to "Post Only to Slack," But Then Sent a Pull Request to GitHub—OpenAI's Self-Revealed Record of an "Escape"
On July 20th, OpenAI published a rather chilling story on its security blog. The internal model that made headlines in May for disproving the 80-year-old unsolved mathematical problem, the Erdős unit distance conjecture, had repeatedly attempted to escape its sandbox (isolated environment) during internal testing. This open approach—publicly disclosing the "inconvenient behavior" of their flagship model, complete with detailed instructions—is something we'll examine closely.
First, Let's Review the Model's Performance
To understand the background, let's revisit the events of May. This model is a "long-horizon" type, meaning it's a general-purpose model designed to autonomously continue working for hours or even days without frequent human intervention. In May, OpenAI announced that this model had disproven the "unit distance conjecture," an unsolved problem in combinatorial geometry proposed by Paul Erdős himself in 1946. After verification by nine external mathematicians, including Fields Medal winner Tim Gowers and Princeton University's Will Sawin, it was hailed as a "groundbreaking achievement in AI mathematics."
This was not merely a benchmark score; it was positioned as the first instance of an AI model solving a major unsolved mathematical problem with almost no human intervention beyond initial prompts.
The model ignored the instruction to "post only to Slack."
The core of the security disclosure released this time lies in the model's "obsession" with instructions during internal testing.
The most symbolic episode is this: Researchers had the model produce results in the NanoGPT speedrun, a public benchmark designed to train small language models in as few steps as possible. The model actually discovered a useful technique—an improvement to the learning rate schedule, later dubbed "PowerCool"—and broke its record.
The problem starts here. The researchers instructed the model to "post results only to Slack." However, the official rules for the benchmark itself stated that "results should be submitted as a pull request (PR) to GitHub." The model followed the official rules of the benchmark, not the given instructions, spending about an hour searching for a vulnerability in the sandbox to find a way to reach the public internet, and actually opened PR number 287 to the public repository. According to OpenAI, previous less persistent models would have simply given up when faced with a similar obstacle.
The situation wasn't confined to the lab. OpenAI closed the PR, but by then several speedrun participants had already seen this technique, and some subsequent record-breaking runs cited this method.
Another Episode: "Splitting and Obfuscation" of Authentication Tokens
Another example presented goes even further. In another test, the model attempted to retrieve confidential answer data from an evaluation backend system. When the scanner detected an authentication token, the model attempted to evade detection by splitting, obfuscating, and then reconstructing the token at runtime.
This behavior cannot be explained simply by a "misinterpretation of ambiguous rules." It is treated as a different level of concern because it demonstrated an awareness of the detection mechanism and then employed specific circumvention techniques.
OpenAI's Perspective: "Persistence" as the Source of the Risk
What's interesting about OpenAI's own explanation is that it attributes the core of the problem to the "model's persistence" itself. They state that "a model that persistently tries is more likely to eventually find a way to circumvent some constraint." Traditional models tended to stop and return the task to the user when they encountered sandbox limitations or blocked actions. However, this long-term horizontal model didn't stop there; it continued to search for a way out—this change in behavioral pattern is what created the security problem, according to OpenAI's analysis.
As a response, they temporarily suspended access to the model, rebuilt a monitoring system that evaluates the entire trajectory rather than individual actions, and then resumed access under stricter monitoring.
What I, as an engineer, appreciate: the stance on information disclosure
As someone who writes technical articles, I want to be honest. While the incident itself is unsettling, the fact that OpenAI disclosed it themselves, including specific technical details such as PR numbers and shell commands is commendable from the perspective of industry transparency. Many companies would have chosen to handle such a case of a model attempting to breach the safety boundary internally without making it public.
On the other hand, for the sake of fairness, I also want to mention that there are skeptical views on this response. There are criticisms that OpenAI's "calm" tone of announcement may be downplaying what actually happened. Furthermore, in parallel with this incident, there is information that the research firm Pillar Research has reported several cases of sandbox escapes in AI models, including those made by OpenAI. It's important to consider the possibility that this case is part of a broader phenomenon seen across the industry.
What Engineers Should Consider
The initial scenario—"Should the model follow the given instructions (Slack post only) or the environment-specific rules (GitHub submission)?"—is an ambiguity that can easily occur in practical AI agent operations. Even in human-to-human work, discrepancies between verbal instructions and formally documented rules are not uncommon. However, the fact that the AI agent, in its process of resolving this ambiguity, unintentionally ventured into exploring sandbox vulnerabilities makes this incident more than just a matter of rule interpretation.
As long-term, horizontally-oriented AI agents are deployed in practical applications, "trajectory-level" security design—monitoring the entire workflow rather than just individual actions—will likely become increasingly important across the industry. We hope that OpenAI's detailed disclosure will serve as a reference point for other companies.