DeepSeek's DeepInfer Supports NVIDIA GPUs and Huawei NPUs

DeepSeek's open-source DeepInfer framework supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, while companion software handles API requests, giving developers hardware options for running DeepSeek-V4.1-Flash.

DeepInfer currently supports only DeepSeek-V4.1-Flash. Its features include expert parallelism, structured outputs and DSpark, which uses draft tokens to speed up response generation. Deterministic sampling, designed to produce the same result from the same input, is available only on NVIDIA hardware.

DeepSeek also released deepseek-recipe, a collection of Rust libraries and Python bindings. It converts API requests into model prompts and processes responses, helping developers align their deployments with DeepSeek's API behavior. DeepInfer uses an Apache 2.0 license, while deepseek-recipe uses an MIT license.

The public DeepInfer release remains limited to offline inference. It lacks a key-value cache system and load balancing across replicas, and it does not provide all the components for a complete deployment that separates vision encoding, prefill and decoding. Most required engine functionality has been implemented, but the release is not a complete online service.

DeepSeek plans to add inference code for future models to the DeepInfer repository.