DeepSeek open-sources DeepInfer, its high-performance inference framework that powers its production services, starting with support for DeepSeek-V4.1-Flash

You can now deploy DeepSeek models using an open-source inference engine that matches its production performance.

DeepSeek AI has open-sourced DeepInfer, its high-performance inference framework designed for large-scale production services. This release enables offline inference and includes core features for serving models at scale. DeepInfer initially supports the DeepSeek-V4.1-Flash model and offers optimized kernels like FlashMLA and DeepGEMM, alongside various parallelism techniques such as Attention DP, balanced CP, and MoE EP + EPLB. It also introduces advanced features like DSpark with confidence-scheduled verification, SWA bounded replay for reduced prefill work, asynchronous scheduling to overlap CPU batch scheduling with device execution, batch-invariant inference for consistent results across different batch compositions, deterministic sampling, and structured output constraints (regex, JSON schema, EBNF). The framework supports NVIDIA GPUs and Huawei Ascend 950 NPUs. For a complete inference service, a KV cache system and VE/Prefill/Decode disaggregation are noted as prerequisites not included in this release.

The open-sourcing of DeepInfer, which powers DeepSeek's production services, democratizes access to a highly optimized inference solution. This move directly challenges proprietary inference frameworks by offering comparable performance and advanced features like structured output generation and optimized kernels. Developers and organizations can now leverage this framework to deploy large language models more efficiently, potentially reducing infrastructure costs and accelerating their AI initiatives without being locked into a specific vendor's ecosystem.

This release signals a growing trend of AI model developers open-sourcing their underlying infrastructure to foster wider adoption and community contribution. By releasing DeepInfer, DeepSeek AI aims to attract developers to its model ecosystem and establish DeepInfer as a de facto standard for efficient LLM inference. The inclusion of support for multiple hardware accelerators, specifically NVIDIA and Huawei, broadens its appeal across different market segments and indicates a strategy to compete effectively in diverse cloud and on-premise environments.