DeepSeek makes DeepInfer open source to cut AI costs

DeepSeek on Tuesday made its in-house proprietary AI inference framework, DeepInfer, open source as it seeks to position itself as the open source provider of choice while appealing to cost-conscious customers.

DeepInfer runs on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs to provide offline inference. The vendor’s choice to run on hardware from Chinese chipmaker Huawei and U.S. AI giant Nvidia comes amid a push from Chinese vendors to rely less on Nvidia. By using both chipmakers, DeepSeek can appeal to its Chinese and Western customers.

The engine is an optimization layer for DeepSeek V4.1 Flash, known for its high-end agentic and coding capabilities and enables the vendor to run the model more efficiently on supported chips. The engine enables the DeepSeek models to generate responses faster and reduce compute costs.

The release of DeepInfer comes as enterprises seek to cut AI inference costs, with many AI models now powering AI agents that require large numbers of tokens, a cost that can be prohibitive.. However, the vendor’s move to open up the inference framework could help its enterprise customers limit those expenses.

Related:Smartsheet takes a people approach with AI adoption

Optimizing cost

“There is a lot of discussion around token optimization,” said Lian Jye Su, an analyst at Omdia, a division of Informa TechTarget. “Token consumption has been going through the roof, and the idea is that if you can turn around a task really quickly and at a low cost, it … helps to optimize your AI infrastructure investment.”

DeepInfer is helpful to enterprises because it accelerates inference processing for DeepSeek models, reducing the number of GPUs required for an inference workload.

“You can turn around your workload faster,” Su said.

While the open source engine will benefit businesses running DeepSeek on private, on-premises infrastructure, enterprises that use other models have access to other open source inference engines, such as Nvidia TensorRT-LLM, which speeds up large language models running on Nvidia GPUs.

The DeepSeek benefit

However, DeepSeek is also moving DeepInfer into the public, open source realm for its own benefit, said Jack Gold, president and analyst at J. Gold Associates.

“From a market messaging perspective, they want to be perceived as the leader in the space, so anything they can do to put stuff out there that says we're exhibiting our leadership is good for them, and it's good for China in general,” he said.

Gold added that the fact that DeepSeek is running the engine on hardware from Chinese chipmaker Huawei is good for the chip provider, as it is trying to build a reputation as a reliable alternative to Nvidia.

Related:Microsoft could hold an advantage with Copilot for Legal

“If it helps, it looks good for the chips. It looks good for the AI model on top,” Gold said.

However, DeepSeek’s continued reliance on Nvidia’s chips also shows that Huawei still lags behind Nvidia in some areas, despite most Chinese vendors' desire to be less reliant on the U.S. AI chip giant, Su said.

Moreover, political tension between the U.S. and China continues to increase, mounting on Tuesday after the U.S. banned federal agencies from using DeepSeek earlier this week, a move China said is an attempt to politicize technology.

Meanwhile, its open source move “tells you that DeepSeek is quite obsessed with innovation,” Su said. “And then quite obsessed with open sourcing those innovation ideas.”

“If you're already using DeepSeek versus other models, the idea is that you are a more cost-conscious enterprise,” he added.

About the Author