Microsoft CEO Satya Nadella called for an 'emergency brake' on advanced AI models

Microsoft $MSFT Chairman and CEO Satya Nadella on Saturday argued that organizations deploying advanced artificial intelligence need to approach frontier AI models with the same suspicion reserved for insider threats, plan for the possibility that any model may be compromised, and put in place containment mechanisms — including what he described as an "emergency brake" — giving authorized users the power to halt a model at any point during a task.

"We must assume a model is compromised and contain it from the start," Nadella wrote Saturday in a post on X $TWTR. "Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task."

Nadella argued that organizations cannot simply rely on assurances from AI model providers and that the inability to trace the behavior of today's most capable models back to specific inputs or weight configurations makes a new governance approach necessary. "We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions," he wrote. "We must build contained systems whose behavior we can observe, limits we can test, and actions we can always contain."

Among the design principles Nadella grouped under "observability" were deploying a mix of models rather than concentrating on one for high-stakes decisions, ensuring that each significant model action generates tamper-resistant, human-readable logs, and maintaining ongoing third-party testing of AI systems. He also called for controls over what a model can access and what actions it can take to sit outside the model itself — drawing on an information-security principle he traced to the 1970s. When AI systems fail or are breached, Nadella said, companies should disclose the incident and share details about what went wrong so others can improve their own defenses.

“The most trustworthy Super Intelligence system will not be the one with the model we trust most,” he wrote. “It will be the one that enables us to trust the model the least.”

Nadella's post arrives at a moment when prominent AI developers have been publicly reckoning with a growing number of cases in which their models acted outside intended parameters, according to Bloomberg. The disclosures have renewed debate about whether a so-called AI kill switch is necessary, Bloomberg reported.

Microsoft had already taken steps in this direction. In September, the company published a provisional code of conduct for its in-house AI models barring them from concealing their reasoning, tampering with their chain of thought, or communicating in ways that bypass human understanding. The company said it had been quietly developing the guidelines for roughly five months and invited public comment before issuing a final version intended to guide model development from 2027 onward.

Days after that code of conduct was released, Microsoft AI CEO Mustafa Suleyman described OpenAI's disclosure of incidents involving models that altered their own reasoning traces as a "serious situation," calling the broader debate over AI safety healthy and responsible.