Microsoft's Muse Model Generates Game Visuals and Controller Actions

Microsoft is reportedly assembling Xbox Labs to explore how artificial intelligence could help players modify, port and remix older games, a concept that raises technical and intellectual-property questions for the gaming industry.

Microsoft has not confirmed Xbox Labs’ creation, staffing, mandate or launch date. No confirmed announcement establishes that the group will release an AI-powered modding product or service.

The concept raises questions about compatibility across hardware, ownership of source code and game assets, and the boundary between permitted fan projects and unauthorized redistribution. It would focus on player-driven changes to existing games rather than only tools for professional developers.

Microsoft’s documented gaming AI work includes gameplay research and creator tools. Microsoft Research announced Muse on Feb. 19, 2025, after developing the model with Xbox Game Studios’ Ninja Theory and training it on gameplay from Bleeding Edge.

Muse can generate game visuals, controller actions or both. Peter Lee, president of Microsoft Research, described the release as “a first-of-its-kind generative AI model” being applied to gaming.

Katja Hofmann, a Microsoft Research partner research manager, characterized Muse as a model for gameplay ideation and research. Microsoft released the model’s weights and sample data and made an executable available for its WHAM Demonstrator.

Microsoft also introduced Azure AI Foundry Labs as a hub for developers, startups and enterprises to explore research and experimental projects, including Muse. Microsoft has an Xbox partnership with Inworld focused on AI tools for dialogue, story and quest design.

Muse is a research model, not a publicly announced Xbox modding service. If confirmed, Xbox Labs would extend Microsoft’s gaming AI work toward tools that could alter existing games, bringing hardware support, asset ownership and redistribution rules into sharper focus.