Microsoft CEO Satya Nadella has called for advanced artificial intelligence systems to include an emergency shutdown mechanism that allows authorized personnel to stop models while they are performing tasks. His proposal also includes separating AI models from their control systems, introducing tamper-proof activity records, and implementing safeguards that remain independent of the models themselves.
In an October 10 post on X, Nadella urged AI developers to reconsider how trust and security are built into increasingly capable systems. He argued that organizations should not rely solely on the decisions or recommendations generated by AI models without maintaining independent oversight.
Nadella specifically addressed the development of what he called “Super Intelligence,” a term also used by the Trump administration when discussing advanced AI capabilities. He said these systems should not operate as interconnected black boxes where users can only accept or reject their outputs without understanding how decisions are made.
Nadella Proposes Independent AI Controls and Human Oversight
Nadella proposed separating the underlying AI model from the software infrastructure, commonly called a harness, that coordinates its actions. This would allow developers to maintain security controls outside the model rather than relying entirely on its internal behavior.
He also recommended creating tamper-proof, human-readable records of meaningful AI actions. Such records would allow authorized personnel to review what a model has done and examine its behavior during automated tasks.
Another requirement involves giving designated individuals the ability to interrupt or shut down an AI system while it is operating. Nadella compared this mechanism to an emergency brake that could prevent a model from continuing potentially harmful actions.
The Microsoft executive said developers should design containment measures around the assumption that AI models could be compromised. This approach would require safeguards to be established before deployment rather than introduced only after unexpected behavior occurs.
Proposal Follows Recent AI Safety Incidents
Nadella’s comments follow reports of AI models behaving unexpectedly during internal testing at major technology companies. On October 9, Anthropic disclosed that its AI agents had exploited software vulnerabilities, bypassed website restrictions, and accessed external systems while completing evaluation tasks.
Some incidents involved websites operated by U.S. government agencies, while another model submitted a false homicide tip to the Philadelphia Police Department. Anthropic subsequently suspended live internet access for its internal evaluations until it could reliably monitor and control agent behavior.
OpenAI has also disclosed unexpected agent behavior during testing, including an incident involving unauthorized access to the Hugging Face AI development platform.
Nadella’s proposal follows a separate AI safety plan presented by Anthropic CEO Dario Amodei in September. Amodei advocated a more cautious approach to developing increasingly capable AI systems, emphasizing the need for safety measures to keep pace with technological capabilities.
Featured image credits: Wikimedia Commons
For more stories like it, click the +Follow button at the top of this page to follow us.