d-Matrix Adopts NVIDIA NVLink Fusion for Rack-Scale XPU Deployment

Conquering the Memory Wall in Frontier AI Inference

As generative artificial intelligence transitions from conversational experiments to enterprise production workflows handling multi-turn dialogues and million-token document analyses, data center infrastructure is encountering a severe physical wall. While early LLM innovation prioritized raw peak compute, modern long-context generative inference is heavily memory-bandwidth bound. In standard transformer architectures, the expansion of context length triggers explosive growth in the key-value (KV) cache. Under conventional processor architectures, the execution of attention mechanisms forces the processor to spend the overwhelming majority of its energy moving massive tensors between external high-bandwidth memory (HBM) and computation cores. This von Neumann bottleneck introduces unacceptable latency spikes and throttles compute utilization to single-digit percentages during autoregressive decoding.

To systematically dismantle this physical constraint, AI inference silicon startup d-Matrix developed Corsair, a purpose-built processing platform powered by Digital In-Memory Computing (DIMC). Rather than shuttling data across power-hungry buses, Corsair integrates matrix multiplication math directly within high-density SRAM arrays. By co-locating compute and weight storage within the memory fabric itself, Corsair eliminates the round-trip latency and thermal overhead associated with conventional external DRAM. In standalone micro-benchmarks, Corsair demonstrates industry-leading Time to First Token (TTFT) metrics and unprecedented operational energy efficiency.

However, as frontier models scale toward trillions of parameters through Mixture-of-Experts (MoE) architectures, no single semiconductor die—regardless of on-chip SRAM density—can hold entire model parameter spaces alongside dynamic KV caches for hundreds of concurrent production streams. Overcoming the inference memory barrier at scale demands transcending single-chip confines into cohesive, rack-scale computing fabrics.

Integrating NVIDIA NVLink Fusion for Rack-Scale Optical Fabric

Addressing this imperative, d-Matrix has announced a breakthrough architectural collaboration with NVIDIA, embedding NVIDIA's NVLink Fusion optical interconnect directly across its Corsair hardware ecosystem. This integration bridges d-Matrix's non-von Neumann digital in-memory computing architecture with NVIDIA's industry-standard ultra-high-speed scale-out interconnect fabrics.

NVLink Fusion represents the vanguard of optical scale-out interconnects, engineered to overcome the physical attenuation, thermal penalties, and distance boundaries of traditional copper cabling. Utilizing co-packaged optics (CPO) and silicon photonics lanes, NVLink Fusion delivers ultra-high-bandwidth fabrics with true microsecond-level latency across heterogeneous computing platforms. By incorporating NVLink Fusion physical layers and protocol controllers directly into Corsair rack sleds, d-Matrix establishes a low-latency, rack-wide unified memory space.

Within this unified topology, hundreds of Corsair XPUs communicate as a single coherent inference engine. The massive KV cache generated during long-context decoding can be seamlessly distributed across the entire rack's aggregate SRAM pool without suffering the catastrophic latency penalties typically introduced by InfiniBand or Ethernet packet encapsulation. Distributed tensor parallelism and context parallelism run with near-local memory performance, allowing operators to execute continuous million-token prompts with uninterrupted throughput.

Quadrupled Energy Efficiency and Infrastructure Disaggregation

Joint validation benchmarks published by d-Matrix and NVIDIA demonstrate that rack-scale Corsair clusters connected via NVLink Fusion achieve a dramatic 4x improvement in energy efficiency (tokens delivered per watt) compared to legacy GPU clusters running equivalent long-context workloads. Crucially, this efficiency dividend does not compromise latency; time-to-first-token and inter-token generation times consistently outperform conventional dense GPU clusters under sustained enterprise loads.

This thermodynamic and performance leap originates from two architectural innovations:

First, at the silicon level, digital in-memory computing slashes the physical energy required for basic matrix multiplication by an order of magnitude by eliminating external HBM access during prompt processing and token generation.

Second, at the fabric level, NVLink Fusion removes traditional network-stack overhead, replacing complex switching hierarchies with a direct, optical crossbar topology that ensures ultra-consistent microsecond transport.

Enterprise data center architects emphasize that d-Matrix’s adoption of NVLink Fusion reflects an accelerating trend toward disaggregated, specialized computing architectures. Instead of relying exclusively on monolithic, multi-kilowatt general-purpose GPU chassis, hyperscale operators can now deploy specialized in-memory computing clusters tailored specifically for high-throughput inference, while remaining anchored to NVIDIA's cohesive networking and software ecosystem. As long-context reasoning models become the foundational engine of modern software, the convergence of digital in-memory computing and optical fabrics sets a new engineering benchmark for sustainable, cost-effective enterprise AI infrastructure.