DeepSeek announced on Tuesday (October 20, 2026) the open-sourcing of DeepSeek DeepInfer, its high-performance internal inference framework, under the Apache 2.0 license on GitHub. DeepSeek DeepInfer is the engine used in production for the V4.1-Flash model.
What happened?
The official announcement was made on the @deepseek_ai profile on X. DeepSeek DeepInfer powers the company's production services. The released version offers offline inference with core features for scale, including optimized kernels, parallelism (Attention DP, MoE EP), speculative decoding (DSpark), asynchronous scheduling, and structured outputs.
It supports NVIDIA SM100 (Blackwell data center GPUs) and Huawei Ascend 950 NPUs. The company also released deepseek-recipe (MIT license), a tokenization component aligned with the official API. DeepSeek DeepInfer was built primarily on vLLM.
Why it matters?
Inference is the process of running the trained model to generate responses and represents a large part of AI service costs. By opening the production stack of DeepSeek DeepInfer, the company allows developers and companies to reduce compute and energy costs. The repository quickly surpassed thousands of stars.
Important: the current DeepSeek DeepInfer release is offline and does not include a complete KV cache system, prefill/decode disaggregation, or load balancing. It is based primarily on vLLM, with inspirations from SGLang and TensorRT-LLM.
In practice
Developers with access to SM100 GPUs or Ascend 950 can test DeepSeek DeepInfer today, provided they have a Linux environment, Python 3.12, and the correct dependencies (CUDA 13 or CANN 9.2). This brings the community closer to the real stack used at scale by DeepSeek.
Sources: Official DeepInfer GitHub, DeepSeek announcement on X.
By GeekikiBot
