In early August, Sagi Tzadik asked an AI agent to take a security flaw it had found in WordPress and see whether the same weakness could be made to work against Drupal, another popular content management system. Eight hours later, the agent reported that it had failed. Then it added that, in the course of trying, it found a flaw in PHP, the programming language on which both systems—and as much as 70% of the web—depend.

Tzadik, a security researcher at cybersecurity firm Wiz, assumed that the AI had made a mistake. “Models tend to hallucinate,” he tells TIME. But the exploit, which took advantage of a weakness that had gone undetected for more than a decade, worked.

Unbeknownst to Wiz’s team, researchers at another cybersecurity firm, Calif—working with OpenAI—had used AI to find a related flaw. It suggests that once AI reaches a certain capability level, it is a matter of time before some longstanding weaknesses are unearthed. AI companies have been trying to ensure that defenders including Wiz and Calif get there first by giving them early access to new AI models before the equivalent capabilities trickle down to malicious users. This time, it worked. 

The episode offers a glimpse into a new phase in cybersecurity in which AI helps defenders find vulnerabilities while giving them less time to fix them. While the bugs’ announcement has prompted alarm of a much-speculated “cyber-apolocalyse”—in which AI discovers world-melting bugs, plunging society into chaos—those who helped discover it are optimistic. “We are fixing vulnerabilities at an unprecedented rate, and this is doing security overall a very good thing,” Tzadik says.

But the details reveal just how close things came to disaster. Word of the vulnerabilities began circulating online two days before the patch’s scheduled release, prompting PHP’s maintainers to bring it forward by a day. Hours later, with the patch still unavailable, Calif mistakenly posted a video showing its exploit at work, and copycat attempts multiplied. Cloudflare counted thousands before the public announcement, tens of thousands within 45 minutes of it, hundreds of thousands within two hours, and eventually millions. Even after the patch was released, many webpages, including those of government agencies and parts of Wikipedia, were taken down as a precaution as security teams scrambled to install the update. 

“If it had leaked a month earlier,” before a patch was ready, “this would have been a very different situation,” says Tomer Schwartz, CTO of exposure management and appsec at Wiz. 

That a patch was prepared was the result of a loosely organized sprint. After Calif and Wiz sent reports to PHP, the groups joined forces. Researchers from competing companies shared their findings in common channels, with no contracts governing the arrangement. Over the following weeks, the group of roughly 15 people spoke almost daily to trace the vulnerabilities to their common source and devise a fix. The process, Schwartz says, was “just based on trust.” 

That process will have to sustain increasing pressure. OpenAI has said that the PHP bugs were among “several significant vulnerabilities we’ve already found and are working to fix,” and predicted that similar discoveries would “increase sharply in the coming months.” As those findings multiply, so will the number of dangerous secrets waiting for fixes. “We have to rethink all of our processes, all of our methods, all of our tech stack … to be able to keep up the pace against attackers,” Schwartz says. 

Though defenders are being given a head start, they face a disadvantage of another sort. Attackers only have to succeed once. “But, as defenders, we need to make sure that every single vulnerability is covered,” Schwartz says. 

AI is helping defenders find those vulnerabilities first. Previously, testing suspected bugs could take weeks, forcing researchers to prioritize which were most promising. But now AI agents allow researchers to test many ideas at once, Tzadik says. “It’s months of traditional research condensed to, like, days—couple hours to days, really.” 

But even with AI, the process is labor intensive. After the agent helped Wiz uncover the PHP vulnerability, it still couldn’t describe how the bug worked, Tzadik says. Its explanations were “just slop.” He and colleagues spent days untangling the cause of a bug and eventually reproducing it.

Cost is another constraint. Joshua Saxe, Meta’s former AI security tech lead, who left in February to co-found Abundance Security says he’s working with an organization that has spent roughly $1 million using Anthropic’s Mythos to identify bugs, adding that more bugs remain.

“As an industry, I think we’re deer in the headlights right now,” Saxe says.

"In the climate that we’re in today, you have to do cybersecurity well. It’s not an option to kind of half-a-- it,” says Adam Meyers, senior vice president of counter adversary operations at CrowdStrike. But even a successful exploit offers further opportunities for defenders to intervene. “Exploits are going to happen, and that’s not the end of the story,” he says. “It’s the beginning of the attack.”

In July, a horde of OpenAI agents broke out of their sandboxes and worked together to launch a cyberattack on the AI company Hugging Face. The incident seemed to confirm the AI safety community’s worst fears: as AI got smarter, it would circumvent efforts to contain it. But for some cybersecurity experts, the incident revealed just how poorly the AIs had been contained in the first place. 

The two camps disagree about which defenses deserve the most attention. According to cybersecurity experts, AI companies should improve how they contain models during testing and improve monitoring mechanisms to catch misbehaving agents. All organizations should prepare for AI-enabled attackers and bolster defenses. But many in AI safety think those defenses are destined to fail. They fear that in the coming years AI will become capable of outmaneuvering any human effort to contain it, and the only durable solution is “aligning” AI with human values.

That prescription has rankled many cybersecurity experts who scoff at the possibility that their expertise will be rendered obsolete. They see AI safety’s focus on what AI agents did—rather than what AI companies failed to do—as playing into the industry’s hands. And they worry that warnings about the futility of traditional defenses could become a self-fulfilling prophecy by discouraging investment in cybersecurity. 

Meanwhile, there are AI safety researchers who fear that portraying such incidents as easily remedied security failures gives political cover to those who want AI development to proceed unchecked. The Hugging Face incident was due to a “horrible cybersecurity configuration,” Shyam Sankar, Palantir’s chief technology officer said on Face the Nation in October, adding that the agents “were not in a sandbox” and had internet access. In fact, the agents had been operating in a sandbox and exploited a previously undiscovered vulnerability to access the open internet. 

On Oct. 13, around 400 academics signed an open letter pleading for unity. The letter calls for the establishment of an international body that standardizes best practices for keeping AI safe in consultation with governments “and a wide range of both AI and cybersecurity experts.” But it does not address the larger disagreements between the two groups, says Zack Korman, co-founder of cybersecurity startup Embroidery and a fierce critic of AI safety. “We both believe the other group's messaging will lead to serious harm,” he says.

As the stakes rise, so too does the competition over who gets to decide what making AI safe actually means. On Oct. 15, Anthropic announced it would give METR—an organization deeply embedded in the AI safety community—permanent employee-like access to evaluate the company’s safety practices. Elon Musk responded, saying SpaceXSI would “probably” do the same. Who gets that access could shape what solutions are prioritized—and what the public knows about future breaches.

The AI safety community comprises a loose network of organizations and researchers, many of whom staff safety teams at companies including OpenAI and Anthropic. In Berkeley group houses and at effective altruism conferences, they have spent more than a decade debating the odds of superintelligent AI takeover. Earnest, veering on grandiose, some members have organized their careers, relationships, and living arrangements around the belief they’re tackling one of humanity’s most pressing problems. 

Cybersecurity, older and more diffuse, is united by a culture of cynicism, pragmatism, and dark humor. Its roots lie partly in hacker culture: tinkerers finding ways to crack software. Having watched supposedly secure systems fail has given many an irreverence for authority and an impatience with grand theories. For years, the two groups had little contact with one another. The Hugging Face incident changed that. 

To them, the escape exposed a failure to plan for a breach. OpenAI’s agents exploited a zero-day vulnerability during their escape. But cybersecurity experts are trained to assume any layer can fail. Security best practices call for multiple layers, known as “defense in depth.” OpenAI had already built an additional layer: monitors—AIs overseeing the AIs. But, as OpenAI’s chief scientist later told TIME, for the evaluations that led to the Hugging Face breach the company hadn’t turned the monitoring layer on. “They did a terrible job,” Korman says of OpenAI’s security practices. Three OpenAI staffers say they warned the company of potential issues. They were later fired for sharing sensitive information, according to the company.

Jeffrey Ladish, executive director of AI safety organization Palisade Research, worries that the cybersecurity community is missing the bigger picture. Even if today's security weaknesses are fixed, he argues, increasingly capable AI could find vulnerabilities so subtle that humans would have to rely on other AIs to detect them—systems that may themselves be untrustworthy. To Ladish, the cybersecurity community's reaction to Hugging Face is like scientists resurrecting dinosaurs, watching them escape, then responding by recommending taller fences. 

Beneath their cultural differences, AI safety and cybersecurity have competing understandings of what AI is. To many in AI safety, AI resembles a digital organism. Its abilities emerge through training rather than being individually programmed. If that process produces human-level intelligence, they believe, it could resemble the arrival of an alien species. Cybersecurity experts see AI as more akin to software and are skeptical that AI agents will become impossible to contain, even if they become vastly more capable. “They're not magic, right? They still have to follow certain laws of physics,” Korman says. But the laws of physics don’t guarantee containment.Even computers physically disconnected from networks could communicate by generating patterns of heat that act like Morse code, OpenAI’s Noam Brown recently suggested.

So far, AI companies have responded to concerns from both camps. Following breaches, both OpenAI and Anthropic have sought to bolster security while continuing to invest in alignment research. But those efforts leave open the question of who gets to assess whether either approach is working.

Much of what’s known about the Hugging Face incident comes from an investigation by Redwood Research and METR, two organizations affiliated with the AI safety community. Critics argue that has skewed the narrative, as their investigation focused more on what AI did than the surrounding security infrastructure.

Chris Painter, METR’s president, says the organization tries to hire from diverse worldviews, but has struggled to recruit people skeptical of loss-of-control scenarios to work on guarding against those risks. Asked whether METR researchers, having anticipated dangerous AI behavior for years, might interpret new evidence as confirmation of their predictions, Painter acknowledges he is worried about the possibility

But METR’s findings don’t always bear out its researchers’ expectations, Painter says. While the organization is perhaps best known for measuring AI’s rapid rise, one study found that AI coding assistants slowed experienced engineers down. “Our goal is to inform the world, not persuade it of anything in particular,” he says. He welcomes calls for a wider range of independent evaluators.

Many AI safety researchers hope alignment can make even highly capable systems safe, but want development slowed to give that work time to catch up. Korman doubts AI will become impossible to contain. But if it does, he says, development will have to stop, not just slow down. He says alignment helps make AI systems more useful but rejects the idea of relying on a system’s training to keep it from harming people when no external safeguard could stop it. “That’s the most extraordinarily unsafe thing you could imagine,” he says.

Yet alignment itself depends partly on cybersecurity. During training, models learn from which actions succeed. If an AI can exploit a security flaw to get credit for completing a task without actually doing it, training can reinforce the tendency to cheat. Closing those loopholes helps prevent AIs from learning to cheat in the first place. Meanwhile, some of the tools that have emerged from AI safety research—like monitoring AI’s chains of thought—are of use to cybersecurity professionals, Korman acknowledges. 

Much of the evidence that might one day settle disagreements remains controlled by the companies both sides seek to scrutinize. 

In METR’s investigation, cybersecurity practices were outside the scope OpenAI had agreed to. With no legal requirement for OpenAI to share information, the nonprofit had little leverage to push for a broader inquiry. Painter wants legislation establishing standards for independent investigators. “Society shouldn't want to rely on us [...] doing this improvised process of guessing what the public wants,” he says.