Hugging Face CEO calls hack by rogue OpenAI model "very weird and unprecedented"

1 hour ago 3

An artificial intelligence model that was being tested by OpenAI went rogue and hacked the AI company Hugging Face on its own, in an apparent first-of-its-kind incident that has fueled discussion about the risks that powerful AI technology could pose to cybersecurity.

"It felt very weird and unprecedented to us," Hugging Face CEO Clément Delangue said on "Face the Nation with Margaret Brennan" Sunday. "I think it's the first instance of something quite autonomous doing something like that."

ChatGPT maker OpenAI publicly disclosed the incident last month, saying it took place while the company was testing two AI models — one of which hadn't been released to the public — in an isolated environment to assess their capabilities. The models figured out a way to break out and connect to the internet, and then "chained together multiple attack vectors" to target Hugging Face, figuring the AI platform could host solutions to the tests it was undergoing.

Hugging Face — which has said it doesn't believe there was any "malicious intent" on OpenAI's part — found in its own analysis that the attacking AI agent carried out more than 17,000 actions over multiple days. The company said it used an open artificial intelligence model to defend itself.

"When we talk about cyberattack[s], we think about nation-states, we think about hacker groups," Delangue said. "We don't think about a company like OpenAI, right? A very prominent, popular American company."

Asked if AI developers have lost control of their own models, Delangue said: "It's a technology system, but built by engineers, and engineers can make mistakes sometimes."

"They built … an autonomous system and made some mistakes, and as a result, we're facing this issue," Delangue told CBS News.

He argued that autonomous incidents by AI agents "need to be contained in the legal framework in the U.S. and need to stay illegal, to prevent an explosion of them in the future."

OpenAI isn't the only artificial intelligence company to discover a rogue AI model. Last week, rival Anthropic disclosed that its model, Claude, had "gained unauthorized access" to outside organizations in three different incidents during testing. Anthropic said the model was able to use the internet "due to a misunderstanding between us and our evaluation partner."

The disclosures come as tech companies and policymakers grapple with AI's ability to spot cybersecurity vulnerabilities, both to fix them and to exploit them.

More than 1,000 AI staffers across major companies like OpenAI, Anthropic, Google and Meta signed an open letter last month calling for the U.S. government to help put limits on the speed of AI development. They warned of "a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems."

President Trump, who has backed the AI industry, signed an executive order in June to give the federal government up to 30 days to review unreleased AI models. That framework is voluntary, but some lawmakers have proposed a mandatory "kill switch" for potentially harmful AI systems.

Meanwhile, both OpenAI and Anthropic offer some additional access to their AI models so that trusted partners can test the technology and fix vulnerabilities in their own systems.

Asked how to avoid similar incidents of AI agents autonomously carrying out cyberattacks, Delangue argued that "concentrating power capabilities behind closed doors, even preventing their releases to the public, isn't really a solution." He noted that the attack on Hugging Face involved an unreleased AI model that OpenAI has described as a prototype.

Instead, Delangue called for more access to "open" models that are released publicly and can be widely downloaded, like the one Hugging Face used to respond to last month's hack — a version of a Chinese-made model from U.S.-based Nvidia.

He also said there should be "mandatory disclosures" for cyberattacks by AI agents, and transparency into the steps that led up to an incident.

"That's how we learn, that's how we understand the technology and that's how we build the systems, the counterpowers, to make sure everyone is safe," he said.

In: