Microsoft CEO Satya Nadella Calls for an AI Emergency Brake

Microsoft CEO Satya Nadella urged AI developers to build systems that authorized people can pause or shut down during a task, calling for stronger controls around AI systems.

In a Saturday post, Nadella called for a reassessment of AI’s trust architecture. He said developers should not treat models as isolated black boxes whose recommendations, answers and actions are simply accepted or rejected.

Nadella urged companies to separate the model from the software that directs its work and to place controls and safeguards outside the model itself. He also said meaningful model actions should be documented with tamper-proof, human-readable evidence.

The emergency-brake approach would require developers to plan for the possibility that a model has been compromised and contain it from the start. Under the framework Nadella described, an authorized person would retain the ability to pause or shut down a system while it is working.

Microsoft has previously described safeguards for frontier artificial intelligence systems, including risk assessments and mitigation measures for capabilities linked to advanced autonomy, cyberoperations, harmful manipulation and loss of control. The company has also announced partnerships with the Center for AI Standards and Innovation (CAISI) in the United States and the U.K. AI Security Institute (AISI) to test advanced models and assess safeguards before deployment.

The company’s broader frontier-AI oversight plans are detailed in its government testing and safeguard efforts, which include external evaluations alongside internal controls.