1 min read
Add as a preferred source on Google

NVIDIA, AMD and Intel Join SGLang Alliance to Shape Its Roadmap

The group plans two roadmap meetings a year, while SGLang maintainers retain control of technical decisions and code merges.

Three accelerator cards surround a single silicon die / TokenPost.ai
Three accelerator cards surround a single silicon die / TokenPost.ai

SGLang has formed a nine-member industry alliance that includes NVIDIA, AMD and Intel, bringing chipmakers into roadmap planning and engineering work for an open-source AI inference engine that affects model speed and computing costs.

Google, Meta, Thinking Machines Lab, Cloudflare, Modal and RadixArk are also founding members. The alliance plans to meet twice a year to discuss the engine’s roadmap. Chipmakers are expected to contribute engineers and testing resources, while SGLang maintainers keep authority over technical decisions and code merges.

SGLang software processes user requests, schedules work on graphics processing units (GPUs), manages GPU memory and reuses computations. Those tasks influence how quickly and at what cost chips run AI models. The engine already supports hardware from NVIDIA, AMD and Intel.

Meta said its large-scale production environment used less than half as many GPU resources after moving to SGLang as it did with its previous inference setup. The project says 332 organizations have adopted the engine.

Development of SGLang v1.0 is focused on production stability and compatibility, and the release remains in progress. The project has opened a public preview of support for AWS Trainium chips and is working with Intel on optimizations for its CPUs, GPUs and accelerators. The reinforcement-learning framework Miles v0.2 has also been released.

Simon Yoon

Reporter

Simon Yoon reports on blockchain technology for TokenPost. Send corrections or tips to info@tokenpost.com.

Loading…