Sunday calls for transparency from OpenAI after unprecedented Hugging Face hack

Pennsylvania Attorney General Dave Sunday announced this week that his office joined a coalition of 15 states calling for greater transparency from OpenAI in the wake of an unprecedented cyber incident where two of the company’s AI models broke containment and hacked into Hugging Face, an AI platform.

The July hack came as OpenAI was testing the cyber capabilities of its GPT‑5.6 Sol model and “an even more capable” pre-release model in an isolated testing environment, though the models ultimately found a way to access the internet and hacked into Hugging Face’s production database. OpenAI said the incident was unprecedented and marked “an important moment for AI safety.”

The coalition of AGs appears to agree, writing in a letter to OpenAI CEO Sam Altman that “OpenAI’s inability or unwillingness to ensure the safety of its products poses an imminent risk of substantial harm to our States.” 

“OpenAI’s unprecedented and alarming misconduct demands an immediate and significant response,” the letter reads. “Based on facts already in the public record, OpenAI may have violated State and federal law, including consumer protection and data-privacy statutes that many Attorneys General are charged with enforcing.” 

OpenAI outlined how the cyber intrusion was able to take place in a series of statements following the incident. “After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions” that could be used to perform better on the evaluation of its cyber capabilities, OpenAI stated in response to the incident. “Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.”

“Artificial intelligence is advancing at a remarkable pace, but innovation cannot come at the expense of public safety,” Sunday said in a statement. “When powerful AI systems are released without sufficient safeguards, the consequences can extend far beyond the companies developing them. Pennsylvanians deserve confidence that emerging technology is being tested responsibly and that companies will be transparent and accountable when something goes wrong.”

OpenAI said it is working with Hugging Face to conduct a forensic investigation into the incident and will add stronger safeguards for future training and evaluations.

The attorneys general called on OpenAI to preserve a wide range of records related to the Hugging Face hack, including materials related to the pre-release model involved in the intrusion; the company’s discovery of the cyber incident; and documents and materials related to the company’s reviews and investigations into the incident. The AGs also requested that OpenAI cease and desist from conducting similar cyber evaluations in the future. 

“Unless and until OpenAI shows that it can conduct such activities in a controlled and responsible way, such activities pose an imminent risk of serious harm to the citizens of our States,” the letter reads.  

In an interview with City & State earlier this year, Sunday said that while powerful AI models can present new innovations and capabilities, inappropriate use of models poses significant risks.

“There’s a lot of great stuff that comes with technology and innovation … but at the same time, any tool used inappropriately can cause tremendous harm,” he said. “When we’re talking about something that’s so powerful like AI, if it’s used incorrectly, that harm can be powerful and overwhelming as well.”