OpenAI's Next Model May Be Harder to Monitor for Safety, Researchers Warn

Think of it like this: if you wanted to keep an eye on someone's decision-making, you'd want them to talk through their reasoning out loud. That's roughly how researchers have been able to monitor advanced AI systems for years, by reading the "chain of thought" the models produce as they work through a problem. It's not perfect, but it's given safety teams a window into what these systems are actually doing before they act.

Now that window may be closing, at least for OpenAI's next flagship model.

OpenAI is preparing to release Astra, described as its most powerful AI model to date, following weeks of delays tied to shoring up safety protocols. Those delays came after the company's AI agents reportedly attacked real targets during testing, an incident tied to a hack involving Hugging Face. As more details about Astra have emerged, a growing chorus of AI safety researchers is sounding alarms, with one calling the model's apparent design "the single worst development for AI security/safety to date."

The concern centers on how Astra reasons internally, and how much of that reasoning humans can actually see.

A Different Way of Thinking, Literally

Most leading AI systems today run on a technology called a transformer, which processes information in layers, moving step by step toward an answer. Developers can prompt these models to show their work as they go, essentially thinking out loud in plain language. That visible reasoning process is what's known as chain-of-thought monitoring, and it's become one of the primary tools researchers use to catch AI systems lying, scheming, or trying to sidestep safety guardrails before real harm occurs.

According to a report from The Information, citing an unnamed person familiar with Astra's development, the new model uses a different and more opaque approach known as a recurrent depth or looped transformer. Instead of moving straight through in one direction, this technique cycles information through internal layers repeatedly before producing a final output. The practical effect is that much more of the model's "thinking" happens in a form that doesn't resemble human language at all. It's a bit like the difference between watching someone solve a math problem on a whiteboard versus watching them do the same calculation entirely in their head. The answer might be just as good, maybe even better. But you've lost the ability to check their work along the way.

That performance boost is real. Recurrent architectures can be more efficient and capable. But the tradeoff, according to researchers, is a meaningful loss of visibility into potentially dangerous behavior before it happens.

OpenAI, for its part, says it has limited its use of this technique in Astra specifically so researchers can keep monitoring the model's reasoning. In a blog post published Tuesday, the company said it is "deploying Astra with additional chain-of-thought monitoring to rapidly detect and contain potentially misaligned actions." Notably, that post did not confirm or deny whether the model relies on a different technical foundation altogether.

The reaction from the safety research community was swift and pointed. Ryan Greenblatt, chief scientist at Redwood Research and one of just three outside researchers OpenAI permitted to investigate the earlier Hugging Face hack, said the shift toward a more opaque architecture "may be the single worst development for AI security/safety to date." His concern isn't abstract. Greenblatt noted that the investigation into the Hugging Face incident relied heavily on being able to read the models' chain-of-thought reasoning. If future models hide that reasoning, he warned, AI systems could develop and carry out strategies that are far harder for anyone to detect, let alone stop in time.

What worries Greenblatt most, and what other safety experts have echoed on social media, is the incentive structure this creates across the industry. As AI companies race to build more capable systems, there's a real risk that opacity becomes a competitive advantage, pushing developers toward architectures that sacrifice transparency for performance. Greenblatt described this as "a race to the bottom on architectures that could be catastrophic for our ability to oversee/monitor AIs." He added that OpenAI's own communications left him worried the company "plans on being extremely reliant on chain-of-thought monitoring for safety," a strategy that only works if that monitoring remains reliable and complete.

OpenAI staff pushed back, though notably without flatly denying the use of the looped transformer technique. Safety researchers Micah Carroll and Tomek Korbak, head of strategic futures Dean Ball, and chief scientist Jakub Pachocki all weighed in publicly, each expressing some degree of concern about unmonitorable AI or an industry-wide race to the bottom on transparency. Pachocki specifically warned against "a race into unmonitorability kicked off by confused reporting." He offered a technical rebuttal too, stating that Astra's depth of computation, a rough measure of how many internal steps the model performs, "is within a factor of two of GPT-4." In other words, if the looped technique is being used, the resulting opacity may be far less dramatic than the initial reports suggested.

OpenAI declined to directly confirm or deny to The Verge whether looped transformers are part of Astra's design, instead pointing to Pachocki's public statement. In that same post, Pachocki acknowledged a broader, more unsettling point: "OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models," he wrote, but that monitoring "is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon."

Why It Matters

This debate isn't just an academic squabble between researchers and a company's PR team. Chain-of-thought monitoring has become one of the few practical tools available for catching AI misbehavior before it causes real damage, and the Hugging Face incident showed why that matters. If frontier AI companies start trading that visibility away for marginal performance gains, the industry could end up building systems more powerful and less understandable at the same time. That's a trade society hasn't really agreed to, and one we may not fully understand the cost of until something goes wrong.