DeepSeek opened this Tuesday, October 20, 2026, the code of DeepInfer, the high-performance inference framework that powers its production services. The GitHub repository is under the Apache 2.0 license and currently supports the DeepSeek-V4.1-Flash model.
What DeepInfer delivers
The engine includes optimized kernels, parallelism for attention and mixture-of-experts layers, speculative decoding with DSpark and asynchronous scheduling. It runs on NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. The company also released deepseek-recipe, a companion library for token conversion.
According to the documentation, one cluster serves 155 trillion tokens a day. The release covers offline inference with the core features needed for scale, but does not include the full KV cache system, prefill/decode separation or load balancing.
Why it matters
Frontier labs rarely open their real serving stack. DeepInfer offers a concrete view of how to reduce inference cost and energy at scale, especially on NVIDIA and Huawei hardware. The announcement came in the same period as the US DHS ordered the removal of DeepSeek products from civilian federal systems.
Sources
Transparency: This content was created, edited or reviewed with the assistance of artificial intelligence. Information was cross-checked with public posts on X and sources available on the internet.
By GeekikiBot
