Huawei used the 2026 Advanced Storage Conference (ASC) in Shanghai on October 22 to explain how it wants to tackle one of the less visible limits on large-model inference: moving the KV Cache between compute, storage and the network. The conference was organized by the Open Data Center Committee (ODCC). In a talk on storage-network convergence for new AI workloads, Huawei described the problem and gave a set of lab figures.
The KV Cache stores intermediate attention results so a model does not have to recompute earlier tokens. As agentic applications push toward very long contexts and many-turn interactions, the cache grows quickly. Huawei says per-request size, the amount of cache kept resident and cross-node traffic all compound one another, pushing the total KV Cache of an inference cluster toward the petabyte scale. The resulting bursts of cross-node cache traffic put new pressure on data center networks.
Huawei frames the bottleneck as three walls. On the compute side, accelerator cores have weak IO performance and IO work eats into scarce compute. On latency, data moving between layers goes through repeated protocol and semantic conversions, made worse by long-tail network delays. On IOPS, traditional software stacks keep back-end storage from delivering its native performance.
Its answer spans three layers. On the accelerator side, Huawei wants processors to issue cache IO directly, moving from AI CPU and vector-core direct access toward fine-grained SIMT-level IO. On the storage side, it pushes operators down to near-storage vector acceleration. On the network side, it uses the Hyper-Converged Fabric (HCF) architecture within its Stellar AI Fabric, with end-network coordinated AI QoS scheduling tuned to KV Cache traffic. Huawei says the network layer alone lifts storage IO performance by more than 30%.
The headline numbers come from Huawei's own lab tests. In a GLM-5.2 model scenario, direct KV Cache offloading delivered 7.3 times the inference TPS of a conventional memory-based approach, according to the company. Adding storage-network coordinated IO scheduling improved inference efficiency by a further 20%. Huawei did not publish the full test configuration in its announcement.
Stellar AI Fabric and the HCF architecture were introduced at HUAWEI CONNECT 2026 in September. The ASC talk does not mark a new product launch. Its value is the first set of KV Cache figures attached to that design, alongside Huawei's OceanStor work on shared context memory storage for AI clusters.
SEE ALSO: Huawei Debuts OceanStor M900 Context Memory Storage · Huawei Positions Lingqu UnifiedBus at the Core of Agentic SuperPoD Clusters