Following its previous announcement of joining the NVIDIA NVLink Fusion ecosystem, enabling its Xeon CPUs to be more closely integrated with NVIDIA GPU-accelerated computing systems, Intel demonstrated a customized hybrid AI rack system during the OCP Summit 2025 earlier. The system uses NVIDIA B200 GPUs, ConnectX-7 network cards, and BlueField-3 DPUs, and even incorporates Broadcom's Tomahawk 5 51.2Tb/s switch chip.
Lock "Separate Inference", Gaudi 3 is responsible for decoding and B200 is responsible for pre-filling
The system demonstrates an architecture called "disaggregated inferencing," the core concept of which is to delegate different stages of AI inference workloads to the hardware that is best at it.
In this architecture, Intel Gaudi 3 focuses on "decoding," where it's designed to be advantageous, while NVIDIA B200 handles "prefill," where it boasts overwhelming performance. Intel claims this clear division of labor significantly improves overall system efficiency.
Intel just took another step on combining forces 🔥 with NVIDIA by integrating their new Gaudi3 rack scale systems together with NVIDIA B200 via disaggregated PD inferencing. Intel claims that compared their B200 only baseline, and inferencing system using Gaudi3 for decode part… pic.twitter.com/jAKin6rgZx
— SemiAnalysis (@SemiAnalysis_) October 18, 2025
Claimed TCO increased by 1.7 times
In terms of specific specifications, this computing rack is equipped with 16 rack modules, each of which contains 2 Xeon CPUs, 4 Gaudi 3 AI chips, 4 ConnectX-7 400 GbE network cards, and 1 NVIDIA BlueField-3 DPU.
Intel claims that the TCO (total cost of ownership) of this hybrid system can be significantly improved by 1.7 times compared to a system that uses only B200 GPUs.
Gaudi's future remains a mystery
However, Intel Gaudi 3 still faces the fundamental problem of immature software support compatibility. At the same time, based on Intel's current internal restructuring, it is not yet certain whether the Gaudi product line will continue to be developed.
On the other hand, NVIDIA is expected to announce the successor to its B200 acceleration system in the first half of next year, making Intel's proposed hybrid system more of a move to clear out Gaudi product inventory. However, it also demonstrates Intel's future XPU hybrid acceleration technology roadmap. In particular, the system design incorporates Broadcom's Tomahawk 5 51.2Tb/s switch chip, rather than directly adopting NVIDIA's Mellanox InfiniBand switch design. This suggests that even with NVIDIA's substantial investment, Intel will not exclusively adopt NVIDIA's design solutions.
