{"id":5402,"date":"2025-05-17T02:12:54","date_gmt":"2025-05-16T18:12:54","guid":{"rendered":"https:\/\/cicserver.com\/pliops-bypasses-hbm-limits-for-gpu-servers-blocks-and-files\/"},"modified":"2025-05-17T02:12:54","modified_gmt":"2025-05-16T18:12:54","slug":"pliops-bypasses-hbm-limits-for-gpu-servers-blocks-and-files","status":"publish","type":"post","link":"https:\/\/cicserver.com\/de\/pliops-bypasses-hbm-limits-for-gpu-servers-blocks-and-files\/","title":{"rendered":"Pliops bypasses HBM limits for GPU servers \u2013 Blocks and Files"},"content":{"rendered":"


\n<\/p>\n

\n <\/p>\n
\"\"<\/a><\/div>\n

<\/p>\n

Key-value accelerator card provider Pliops has unveiled the FusIOnX stack as an end-to-end AI inference offering based on its XDP LightningAI card.<\/p>\n

Pliops\u2019 XDP LightningAI<\/a> PCI card and software augment the high-bandwidth memory (HBM) memory tier for GPU servers and accelerate vLLMs on Nvidia Dynamo by 2.5x. UC Berkeley\u2019s open source virtual large language model (vLLM) library for inferencing and serving uses a key-value (KV) cache as a short-term memory for batching user responses. Nvidia\u2019s Dynamo<\/a> framework is open source software to optimize inference engines such as TensorRT LLM and vLLM. The XDP LightningAI<\/a> is a PCIe add-in card and functions as a memory tier for GPU servers. It is powered by ASIC hardware and software, and caches intermediate LLM process step values on NVMe\/RDMA-accessed SSDs.<\/p>\n

\n
\"Pliops<\/figure>\n<\/div>\n

Pliops says GPU servers have limited amounts of HBM. Its technology is intended to deal with the situation where a model\u2019s context window \u2013 its set of in-use tokens \u2013 grows so large that it overflows the available HBM capacity, and evicted contexts have to be recomputed. The model is memory-limited and its execution time ramps up as the context window size increases.<\/p>\n

By storing the already-computed contexts in fast-access SSDs, retrieving them when needed, the model\u2019s overall run time is reduced compared with recomputing the contexts. Users can get more HBM capacity by buying more GPU servers, but the cost of this is high, and bulking out HBM capacity with a sub-HBM storage tier is much less expensive and, we understand, almost as fast. The XDP LightningAI card with FusIOnX software provides, Pliops says, \u201cup to 8x faster end-to-end GPU inference.\u201d<\/p>\n

Think of FusIOnX as AI stack glue for AI workloads. Pliops provides several examples:<\/p>\n