회사 소식 Huawei’s OceanStor KV cache storage for hyper-scale AI data centers

Huawei has announced the OceanStor M900, a scale-out, all-flash storage cluster delivering up to a 64 PB KV cache base tier for its Atlas 960 SuperPoDs; rack-scale AI accelerators comparable to Nvidia’s SuperPODs.

Nvidia has defined a reference design integrating its GPUs, BlueField NICs, Spectrum-X switches, and AI software with storage systems, supporting GPUDirect, RDMA, and KV cache extension via its Dynamo software and CMX scheme. This multi-tier architecture uses the GPU’s high-bandwidth memory (HBM) as the fastest top tier, followed by the associated X86 servers’ DRAM as level 2, the server’s local SSDs as L3, and intelligent BlueField-4 NIC-connected NVMe SSDs within a flash storage server serving as L3.5. These can reach a GPU’s HBM in a single network hop with microsecond-class latency.

Huawei is now challenging Nvidia’s SuperPOD design with its own SuperPoDs built for hyperscale AI data centers. The systems target 10-trillion parameter models and million-plus token context windows, meaning a single accelerator’s high-bandwidth memory cannot hold all required key-value tokens, necessitating a caching mechanism.

The company states one Atlas 960E SuperPoD can scale up to 4,096 NPUs (Neural Processing Units), delivering 8 EFLOPS of FP8 compute performance, up to 1 petabyte of HBM capacity and a 256 TB unified memory pool. Deploying 5,500 Hi-ONE units equipped with UnifiedBus (NPO) cuts power consumption by over 550 kilowatts compared to the 48,000 800G optical modules traditionally required to interconnect all NPUs. It also doubles the system’s fault-free operating time and reaches 99.8 percent system availability.

Huawei OceanStor M900
The SuperPoD adopts a multi-tier KV caching framework, where the M900 supplies petabyte-scale KV cache for the L3.5 layer. Each cluster delivers up to 4 PB of shared L3.5 KV cache capacity and 40 TB/sec of aggregate bandwidth over optical networking for this tier, providing multiple terabytes of KV cache capacity per NPU. Huawei claims that under typical AI inference workloads, this architecture doubles the inference cluster’s token throughput and cuts time to first token (TTFT) in half.

The firm notes the 40 TB/sec bandwidth is 1.5 times higher than competing solutions, without explicitly naming vendors. It is understood to generally refer to DDN, Everpure, IBM (Storage Scale), MinIO and VAST Data systems whose cluster bandwidth ranges between 10–25 TB/sec.

The M900 features an integrated design combining CPU, network controller and NAND controller in one unit, enabling SuperPoD NPUs to establish a direct, one-hop link to SSDs. This reduces access latency from milliseconds to roughly 60 microseconds.

David Wang, Huawei Deputy Chairman of the Board and Rotating Chairman, said: “OceanStor M900 also uses hybrid media and an optimized retention algorithm, extending SSD read/write lifespan by 16-fold. This ensures a higher KV cache hit rate alongside long-term stability and reliability from the ground up.”

In further details, Huawei states the M900 features KV-aware adaptive storage technology that predicts the expected lifetime and value of each segment of KV cache data. Based on these predictions, it schedules and distributes data across different media tiers: on-chip memory, DRAM, and SSDs. Huawei says this optimized placement and retention strategy enables SSDs to sustain up to 24 drive writes per day (DWPD), a notably high figure, and boosts SSD endurance 16 times, supporting three years of stable operation with fewer drive replacements.

It remains too early for Huawei to release M900 datasheets and technical briefings, so details such as node rack unit size, controllers, drive quantity and capacity, cluster node count and other specifications are not yet available.

NPU Footnote
NPUs are Huawei Ascend Neural Processing Units which, Huawei says, are built natively for AI (unlike GPUs that originated as graphics chips). They adopt Huawei’s Da Vinci architecture with Cube (matrix) cores and Vector cores, and support low-precision formats including FP8 and FP4 to accelerate inference and training for large models. When assembled into massive systems via Huawei’s UnifiedBus all-optical interconnect, they can scale to thousands or even hundreds of thousands of NPUs operating as one logical machine. Recent offerings such as the Atlas 350 (using Ascend 950PR) and upcoming Atlas 960 SuperPoDs are positioned as alternatives to Nvidia GPUs within the Chinese market.

Beijing Qianxing Jietong Technology Co., Ltd.
Sandy Yang/Global Strategy Director
WhatsApp / WeChat: +86 13426366826
Email: yangyd@qianxingdata.com
Website: www.qianxingdata.com/www.storagesserver.com
Business Focus:
ICT Product Distribution/System Integration & Services/Infrastructure Solutions
With 20+ years of IT distribution experience, we partner with leading global brands to deliver reliable products and professional services.
“Using Technology to Build an Intelligent World”Your Trusted ICT Product Service Provider!