Tag: Gaudi

In November, news broke that Intel intended to acquire AI startup SambaNova for $5 billion, hoping to accelerate its AI transformation through RDU architecture design. Now, reports suggest the actual deal may have shrunk to $16 billion. According to Bloomberg News, compared to the previous $5 billion acquisition, this latest report indicates the deal may be worth only around $16 billion. If true, this price would be a significant loss for SambaNova. Founded in 2017, the startup was valued at over $5 billion during its 2021 funding round, reflecting the survival pressures and valuation corrections faced by second-tier AI chip startups in the current AI hardware market dominated by NVIDIA. The role of Intel's current CEO, Lip-Bu Tan, behind this potential deal is intriguing. SambaNova already boasts a prestigious investor lineup, including SoftBank, Google Ventures, and Intel's own Intel Capital. As for Chen Liwu's personal investment company, Walden International, it was also one of the early investors. Chen Liwu himself was appointed as the executive director of SambaNova last May. Therefore, this deep connection is seen by outsiders as a key catalyst that accelerated the negotiations between the two parties. What to do with it? Targeting the "inference" and "complete system" market, SambaNova's core competitiveness lies in its DataScale system and SN40L processor, focusing on high-performance generative AI inference and fine-tuning. Unlike NVIDIA, which focuses on general-purpose GPU computing, SambaNova adopts a software-defined hardware architecture. Therefore, for Intel, acquiring SambaNova has several strategic implications: • Strengthening inference computing power: As the Gaudi series struggles to compete with the H100/B200 in the market, SambaNova technology can provide Intel with a differentiated advantage in specific areas (such as large language model inference). • Complete System Solutions: SambaNova doesn't just sell chips; it excels at providing complete solutions encompassing hardware, compilers, and model-as-a-Service (MAS) services. This is precisely the missing piece in Intel's current attempt to transform into a "system foundry" and "enterprise AI service" provider. Analysis: Acquisitions are easy, integration is difficult. I believe Intel's move is clearly aimed at quickly closing the technological gap with its competitors. $16 billion is not a small sum for Intel, which is currently facing financial constraints, but it may be a necessary gamble. However, Intel's past acquisition record in the AI ​​field is not perfect...

Following its earlier announcement of joining the NVIDIA NVLink Fusion ecosystem, enabling tighter integration of its Xeon CPUs with NVIDIA GPUs for accelerated computing, Intel showcased a custom hybrid AI rack system at OCP Summit 2025. This system utilizes an NVIDIA B200 GPU, ConnectX-7 network card, and BlueField-3 DPU, and even incorporates Broadcom's Tomahawk 5 51.2Tb/s switch chip. Focusing on "disaggregated inferencing," the system employs a "disaggregated inferencing" architecture. The core concept is to assign different stages of the AI ​​inferencing workload to the hardware best suited for them. Under this architecture, the Intel Gaudi 3 focuses on its design-advantageous "decoding" task, while the NVIDIA B200 handles its overwhelmingly powerful "prefill" task. Intel claims that this clear division of labor significantly improves overall system efficiency. Intel just took...

During Computex 2024 this year, Intel showcased its sixth-generation Xeon processors, codenamed "Sierra Forest," manufactured using Intel's 3-processor and featuring an all-E Core design. Following the demonstration of the Gaudi 3 AI accelerator suite, Intel recently announced the launch of its sixth-generation Xeon processors with a P Core design. Intel also emphasized that the Gaudi 3 AI accelerator suite delivers twice the performance at the same cost compared to NVIDIA's H100 acceleration solution when performing LLaMa 2 70 billion parameter model inferences. Intel explained that the sixth-generation Xeon processors with added P Cores are designed for handling computationally intensive workloads while maintaining performance advantages. Compared to the previous generation, the sixth-generation Xeon processors offer twice the performance, with an increased core count, doubled memory bandwidth, and AI acceleration features embedded in every core, thereby meeting the AI ​​performance demands of edge computing, data centers, and cloud environments. The Gaudi 3 AI Accelerator Kit is optimized for large-scale, automatically generated AI. It features 64 Tensor Processing Cores (TPCs) and 8 Matrix Multiplication Engines (MMEs) to accelerate deep neural network computations. It also includes 128GB of HBM2e memory for training and inference, and 24 200 Gb Ethernet ports for scalable networking. Gaudi 3 is compatible with the PyTorch framework, Hugging Face Transformer models, and diffuser models. Intel has partnered with IBM to deploy the Gaudi 3 AI Accelerator Kit on IBM Cloud, allowing IBM customers to use AI and scale their applications while improving computing performance and reducing total cost of ownership (TCO). Intel is currently offering a preview of its sixth-generation Xeon processors through its Tiber Developer Cloud platform, allowing customers to conduct technical evaluations and testing. Select customers will have early access to the Gaudi 3 AI Accelerator Kit to verify the compatibility of AI model deployments. The Gaudi 3 cluster is expected to be available starting next quarter and will be suitable for large-scale production deployments. Intel also announced that it will face real-time monitoring as its automated generative AI solutions progress from prototyping to production readiness...

At Vision 2024, Intel announced the Gaudi 3 AI Accelerator, which offers approximately four times the AI ​​performance in BF16 computing compared to its predecessor, along with 1.5 times the memory bandwidth and 2 times the network bandwidth. This enables larger-scale AI computing and accelerates AI training for large natural language models and multimodal models, as well as improving inference efficiency. Justin Hotard, Intel Executive Vice President and General Manager of Data Center and AI Solutions, stated, "The AI ​​market is rapidly evolving, yet significant gaps remain between products. Feedback from both customers and the broader market reflects a desire for more choices. Enterprises must weigh factors such as availability, scalability, performance, cost, and energy efficiency. Intel Gaudi 3 stands out as a new option for generative AI, offering a perfect combination of cost-effectiveness, system scalability, and time-saving advantages." The Gaudi 3 accelerator is designed for high-performance, large-scale AI computing needs, manufactured using a 5nm process. Its design allows all engines to be activated simultaneously, thereby increasing overall computing speed, including the Matrix Multiplication Engine (MME), Tensor Processing Core (TPC), and Network Interface Card (NIC), enabling faster, more efficient deep learning computing and scalability. Leveraging a dedicated heterogeneous computing engine design, the Gaudi 3 accelerator comprises 64 AI-defined and programmable tensor processor cores, along with eight matrix multiplication engines. Each Gaudi 3 matrix multiplication engine can perform 64000 parallel operations, supporting highly efficient complex matrix calculations and various data types, including FP8 and BF16. To meet the demands of large-scale natural language model computations, the Gaudi 3 accelerator features 128GB of HBMe2 memory, 3.7TB of memory bandwidth, and 96MB of on-board static random access memory. Furthermore, each Gaudi 3 accelerator integrates 24 Ethernet ports with a throughput of up to 200GB, enabling efficient expansion and support for large computation sets, supporting expansion from a single node to over 1000 nodes. Furthermore, the Gaudi 3 accelerator integrates the PyTorch framework via Gaudi software and can utilize optimized models hosted by the Hugging Face community. This supports common generative AI frameworks, improving the ease of AI deployment and productivity, while also enabling rapid model migration to different hardware. In addition to the standard version, the Gaudi 3 accelerator also offers a PCIe version, packaged with a 600W TDP and integrating 128GB of memory, supporting a data transfer bandwidth of 3.7TB per second. Compared to NVIDIA's H100 accelerator, Intel boasts that its Gaudi 3 accelerator can reduce training time by 50% for Llama2 7B and 13B parameter scale models, and GPT-3 175B parameter scale models. It also offers a 30% improvement in inference speed and a 50% increase in inference throughput for Llama 7B and 70B parameter scale models, and a 40% reduction in inference power consumption. Furthermore, it exhibits a significant performance advantage in inference over longer input and output sequences.

At the "AI Everywhere" event in New York, Intel announced the "Meteor Lake" Core Ultra series laptop processors and the 5th generation Xeon Scalable server processors, codenamed "Emerald Rapids." Intel CEO Pat Gelsinger stated that artificial intelligence (AI) will become a major driver of the global economy, emphasizing Intel's continued development of technologies to enhance enterprise computing and AI capabilities. The event also showcased the new Gaudi 3 AI accelerator. Gelsinger explained that AI innovation will account for one-third of global economic growth in the future, and AI will be integrated into various computing platforms, including cloud and edge devices, with self-generating AI becoming a more significant development. Intel is currently enabling AI computing capabilities in cloud, server, edge, and even laptop devices through its processors, making AI applications more ubiquitous and connecting more application possibilities through an open ecosystem. At the event, Pat Gelsinger also showcased the new Gaudi 3 AI accelerator, which is expected to be officially launched in 2024. It will be able to accelerate the deep training of large-scale AI models, reduce ownership costs, and promote the growth of automatically generated AI technology applications.