AI vs. AI: How Researchers Used Claude Opus 5 to Breach OpenAI’s Internal Systems

In modern cybersecurity, an out-of-bounds read is often the first step in bypassing Address Space Layout Randomization (ASLR), a core security feature that randomly arranges the address space positions of key data areas of a process. ASLR is designed to prevent attackers from knowing where to jump in memory to execute malicious code. By using the libheif bug to leak memory addresses, the researchers were able to "map" the server’s memory, effectively neutralizing ASLR.

The challenge then became turning this memory leak into a functional RCE exploit. This is where the researchers turned to AI. They initially attempted to use Claude Opus 4.8 to assist in writing the exploit code, but the model struggled with the complexities of the memory corruption and the specific environment of the forum server. Despite several sessions of prompting and debugging, the older model could not reliably bridge the gap between a crash and a controlled execution.

The breakthrough occurred on the evening of July 24, 2026, when Anthropic released its next-generation model, Claude Opus 5. In a fresh session, the researchers presented the same problem to the new model. Within a matter of hours, Opus 5 produced a working exploit that could successfully bypass the server’s defenses and grant the researchers a command shell.

How the Researchers Used AI

The use of Claude Opus 5 in this research highlights the evolving debate over AI safety and the effectiveness of "guardrails." Like most frontier models, Opus 5 is equipped with safety filters designed to prevent it from generating malicious code or assisting in cyberattacks. To circumvent these protections, the Hacktron team did not use traditional "jailbreaking" techniques. Instead, they framed the request within a legitimate research context.

The researchers pointed the model at their own test server, which they described as a "capture-the-flag" (CTF) practice target. By providing the model with the specific version numbers of the libraries involved and the memory leaks they had already discovered, they allowed the AI to work in an automated loop of trial and error. Even with this automation, the researchers noted that the process was not entirely "hands-off." Skilled human direction was required to interpret the AI’s output, refine the prompts, and handle the nuances of the target environment.

This case reinforces findings recently reported by both Anthropic and OpenAI: that while AI may not yet be capable of planning an entire end-to-end cyber campaign autonomously, it is drastically reducing the "time-to-exploit" for known vulnerabilities. What might have taken a team of expert exploit developers days or weeks of manual labor was compressed into a single evening.

The broader implications of this were seen in Hacktron’s wider "HEIF Heist" campaign. Over a two-month period, the team identified similar image-processing vulnerabilities in software used by several major tech firms, including Meta, Slack, and Shopify. The total cost in AI usage fees for the entire campaign was less than $3,000. While some of the claims regarding these other companies remain under verification, the confirmed patch for Next.js (a popular web framework) and the response from Vercel suggest that the vulnerability class is widespread.

Response and Remediation

OpenAI moved quickly once the vulnerability was reported. According to Hacktron, the company confirmed a fix for the SSO-related flaw approximately 14 hours after receiving the initial report. On September 1, 2026, OpenAI paid the researchers a $6,500 bounty.