This AI SSD tech makes 8 RTX 5090s perform like 46 GPUs in inference

GenStorAIGE AI90 shifts AI memory beyond traditional GPU HBM limitations using SSDs PT200Z SSD supports constant cache updates during demanding inference workloads efficiently Eight RTX 5090 GPUs gain dramatically larger effective inference memory capacity GenStorAIGE has introduced its AI90 inferen

<![CDATA[ <article> <ul><li><strong>GenStorAIGE AI90 shifts AI memory beyond traditional GPU HBM limitations using SSDs</strong></li><li><strong>PT200Z SSD supports constant cache updates during demanding inference workloads efficiently</strong></li><li><strong>Eight RTX 5090 GPUs gain dramatically larger effective inference memory capacity</strong></li></ul><p>GenStorAIGE has introduced its AI90 inference acceleration platform at WAIC 2026, taking a storage-centric approach to expanding effective AI memory capacity.</p><p>Rather than depending solely on <a href="https://www.techradar.com/news/computing-components/graphics-cards/best-graphics-cards-1291458">GPU</a> high-bandwidth memory, the platform incorporates PCIe Gen5 solid-state drives directly into the memory hierarchy itself.</p><p>This allows portions of the Key-Value Cache used by large language models to sit outside GPU memory entirely.</p><h2 id="a-three-tier-memory-architecture-built-around-ssd-offloading">A three-tier memory architecture built around SSD offloading</h2><p>AI90 combines HBM, system DRAM, and <a href="https://www.techradar.com/news/best-solid-state-drives-ssds">SSD</a> into a unified three-tier memory structure for handling inference workloads.</p><p>By transparently offloading KV Cache data onto SSDs, the platform reduces pressure on GPU memory while supporting significantly larger workloads and longer context windows.</p><p>According to GenStorAIGE, this architecture cuts first-token latency from several seconds down to sub-second response times in supported configurations.</p><p>That represents up to a 50x improvement, alongside throughput gains reaching 5.1x and a roughly 39% reduction in GPU memory usage.</p><p>Combined with intelligent peer-to-peer GPU communication, the company states AI90 can accelerate inference by up to 5.8x on systems running eight Nvidia GeForce RTX 5090 cards.</p><p>That multiplier effectively allows an eight-card setup to behave closer to a 46-GPU cluster during sustained inference tasks.</p><p>The architecture also supports context windows exceeding 128,000 tokens, enabling far larger document processing and conversation handling without exhausting available memory.</p><h2 id="the-pt200z-ssd-handles-the-intensive-write-demands-behind-the-system">The PT200Z SSD handles the intensive write demands behind the system</h2><p>To support continuous write workloads generated by constant KV Cache updates, GenStorAIGE paired AI90 with its new PT200Z AI SSD.</p><p>Built using pSLC NAND flash and connected through a PCIe Gen5 x4 interface, the drive delivers sequential read speeds reaching 14.8 GB/s.</p><p>Random read performance hits approximately 3.1 million IOPS, while read latency sits at just 54 microseconds.</p><p>Write latency drops even further to 10 microseconds, supporting the rapid cache updates AI90's architecture depends on constantly.</p><p>Endurance ratings reach up to 100 drive writes per day, a figure suited for sustained enterprise AI workloads with constantly shifting cache data.</p><p>This design reflects a broader shift across AI infrastructure toward memory tiering, as <a href="https://www.techradar.com/computing/artificial-intelligence/best-llms">LLMs</a> increasingly outgrow the practical limits of GPU HBM alone.</p><p>Integrating extremely fast SSD storage into inference pipelines offers one method for scaling context length without requiring additional GPUs or larger HBM configurations.</p><p>Whether the performance claims hold outside controlled testing conditions remains genuinely unverified at this stage.</p><p>As with most vendor announcements, these performance figures come directly from GenStorAIGE and still require independent benchmarking across varied real-world AI workloads.</p><p>Via <a href="https://www.guru3d.com/story/genstoraige-ai90-reduces-first-token-latency-by-up-to-50x/" target="_blank" rel="nofollow">The Guru of 3D</a></p><figure class="van-image-figure inline-layout" data-bordeaux-image-check ><div class='image-full-width-wrapper'><div class='image-widthsetter' style="max-width:676px;"><p class="vanilla-image-block" style="padding-top:31.51%;"><img id="diM9tpwF2Lz85R8q85CT78" name="tr-g_news" alt="Google logo on a black background next to text reading 'Click to follow TechRadar'" src="https://cdn.mos.cms.futurecdn.net/diM9tpwF2Lz85R8q85CT78.jpg" mos="" align="middle" fullscreen="" width="676" height="213" attribution="" endorsement="" class="inline"></p></div></div></figure> </article> ]]>