Science & Tech · AI · 21 hrs ago
DeepSeek open-sources AI inference framework for Nvidia and Huawei chips
Chinese AI company DeepSeek has open-sourced DeepInfer, a framework it developed to run AI models.
It supports Nvidia GPUs and Huawei Ascend processors, and currently works with DeepSeek's V4.1-Flash model.
Inference is the process of running an AI model to respond to requests, and frameworks can help make that process more efficient.
DeepInfer includes tools for handling large workloads and producing structured outputs.
DeepSeek says the framework can improve speed while cutting costs and energy use, which matter to businesses running AI services.
The company also open-sourced a component for processing API requests and model inputs and outputs.
DeepSeek plans to release more inference code and improvements for future models through the project.
DeepSeek released DeepInfer, an open-source framework for running AI models, on October 20.
The framework supports Nvidia GPUs and Huawei Ascend processors.
It offers distributed inference, speculative decoding and structured output.
DeepSeek said the release aims to improve throughput and latency while reducing computing costs and energy use.
The framework currently supports DeepSeek-V4.1-Flash-1017, and DeepSeek plans to release further inference code and optimizations.
- Who
- Chinese artificial intelligence start-up DeepSeek.
- What
- It released DeepInfer, an open-source framework for running AI models.
- When
- October 20, 2026.
- Where
- Not stated.
- Why
- DeepSeek said the framework is intended to improve throughput and latency while reducing computing costs and energy consumption.
This story does not have two clearly opposing sides.
DeepSeek released its DeepInfer inference framework.
DeepSeek plans to release further inference code and optimizations for future models through the project.
- Framework
- DeepInfer
- Release date
- October 20, 2026
- Supported chips
- Nvidia GPUs and Huawei Ascend processors
- Current model support
- DeepSeek-V4.1-Flash-1017
4 sources
What people are saying
light and darkness