DeepSeek Opens DeepInfer Code for Offline Model Inference

DeepSeek has released its DeepInfer inference framework as open source, making offline model inference code available while leaving out components needed to operate a complete online service.

DeepSeek has used DeepInfer to serve DeepSeek-V4.1-Flash at scale. The public release has a narrower scope than a complete inference service.

The published code supports offline inference but does not include a KV cache system or load balancing. Both are among the components needed for a complete service. The release therefore gives developers access to part of DeepInfer without providing every component used to run inference at scale.

DeepSeek plans to use the project to publish inference code for future models and continue adding optimizations. Those updates are planned; they are not part of the current release.