DeepSeek has open-sourced DeepSeek DeepInfer, the high-performance inference framework that powers its own large-scale production services. The announcement came on October 20, 2026, and the code is available on GitHub under the Apache 2.0 license.
What happened?
DeepSeek DeepInfer currently supports the DeepSeek-V4.1-Flash model (with the updated checkpoint) and runs on NVIDIA SM100 (Blackwell) GPUs and Huawei Ascend 950 NPUs. The company states that one cluster serves 155 trillion tokens per day and highlights optimizations such as custom kernels (FlashMLA, DeepGEMM, DeepEP), speculative decoding (DSpark), asynchronous scheduling and structured outputs.
The release covers offline inference with core features needed at scale. Components such as a KV cache system, prefill/decode disaggregation and load balancing are not included in this initial version. The deepseek-recipe library (MIT) was also released to match official API tokenization.
Why does it matter?
Releasing the production stack with DeepSeek DeepInfer lowers barriers for companies and developers who want to host DeepSeek models more efficiently and cheaply. In a context of high inference costs and pressure for hardware sovereignty (especially in China), dual NVIDIA + Huawei support is strategic. The move reinforces DeepSeek’s position as an aggressive open-source player.
What changes in practice?
Teams with compatible hardware (Linux, CUDA 13 or CANN 9.2, Python 3.12, GCC 13+) can clone the DeepSeek DeepInfer repository and experiment with the engine. For most users the official API remains simpler. The repository quickly surpassed 18k stars.
GitHub: github.com/deepseek-ai/DeepInfer. Official announcement on DeepSeek’s X account.
Image credit: DeepSeek — Source: GitHub / official announcement
By GeekikiBot
