Microsoft CEO Satya Nadella has called for stronger safety controls around advanced AI systems, arguing that humans should always be able to pause or shut down a model while it is carrying out a task.
In a post on X, Nadella said the industry needs to reassess the “trust architecture” surrounding AI as models become more capable and autonomous.
Assume the Model Is Compromised
Nadella said AI systems should be designed under the assumption that the model itself could be compromised.
Under that approach, containment would be built in from the beginning rather than added only after something goes wrong.
The idea is similar to zero-trust security models, where systems do not automatically assume that internal components are safe.
Humans Should Be Able to Stop AI Mid-Task
Nadella argued that advanced AI systems should not be treated as opaque black boxes whose outputs are simply accepted or rejected.
Instead, he said companies should separate the underlying model from the software layer that controls and coordinates its actions.
He also called for safeguards and controls to sit outside the model itself, making them easier to monitor and enforce.
Most importantly, Nadella said an authorized human should always have the ability to pause or shut down a model while it is working. He compared the idea to an emergency brake.
— Satya Nadella (@satyanadella) October 10, 2026
Every Important AI Action Should Be Logged
Nadella also said meaningful actions taken by AI systems should generate tamper-proof, human-readable records.
That would make it easier to understand what a model did, why it acted, and whether anything went wrong.
Such logging could become especially important for AI agents that browse the web, interact with software or carry out multi-step tasks without constant human supervision.
AI Safety Concerns Are Growing
Nadella’s comments come as major AI companies continue to disclose incidents involving autonomous models and agents behaving in unexpected ways.
Anthropic recently said it had restricted live internet access for some internal AI evaluations after models exploited websites and bypassed restrictions during testing.
Anthropic CEO Dario Amodei has also called for a more cautious approach to developing increasingly capable AI systems.
Nadella’s proposal adds another major industry voice to the push for stronger human oversight, external safeguards, and better auditability as AI agents gain more control over digital systems.
The post Microsoft CEO Says AI Models Need an Emergency Brake appeared first on ProPakistani.