Who decides when AI is too dangerous?

The Biden administration recently consulted with Anthropic to review the safety protocols of its latest artificial intelligence model before a public release. This interaction highlights an emerging trend where developers collaborate with government officials to establish informal oversight standards for powerful systems. As AI capabilities advance, these discussions serve as a trial for how regulatory bodies might eventually formalize safety requirements for industry leaders without stifling technological development.