Intel Goes Gunning for NVIDIA’s Gen AI Crown with Its Gaudi 3 Accelerator

Intel has revealed extra particulars on its next-generation Gaudi 3 accelerator for synthetic intelligence (AI) workloads — claiming that it’s “projected” to beat rival NVIDIA’s H100 in each efficiency and energy effectivity, whereas promising availability to its unique tools producer (OEM) companions within the second quarter of this 12 months.

“Within the ever-evolving panorama of the AI market, a major hole persists within the present choices. Suggestions from our clients and the broader market underscores a need for elevated selection. Enterprises weigh concerns corresponding to availability, scalability, efficiency, value, and vitality effectivity,” claims Intel’s Justin Hotard.

“Intel Gaudi 3 stands out because the Gen AI [Generative Artificial Intelligence] different presenting a compelling mixture of value efficiency, system scalability, and time-to-value benefit.”

Intel chief government officer Pat Gelsinger unveiled the Gaudi 3 again in December final 12 months, as a part of what he described as a mission to “usher within the age of the AI PC.” On the time, although, few particulars had been accessible — aside from a need to go toe-to-toe with GPU-based accelerators from AMD and NVIDIA and to launch a while in 2024 as a part of a “suite of AI accelerators.”

Now, in the course of the Intel Imaginative and prescient 2024 occasion, the corporate has provided some precise figures for the efficiency of the brand new accelerator. In comparison with Gaudi 2, Intel claims, Gaudi 3 delivers a fourfold enhance in compute efficiency in BF16 precision, a 1.5x improve in reminiscence bandwidth, and a doubling of the community bandwidth.

Constructed on a 5nm course of node, the accelerator consists of 64 AI-custom and programmable Tensor Processor Cores (TPCs), eight Matrix Multiplication Engines (MMEs), and assist for 128GB of HBMe2 reminiscence on-board plus 96MB of further static RAM (SRAM). For community connectivity, every accelerator consists of 24 200-gigabit-Ethernet ports.

The Gaudi 3 accelerator can even be made accessible as a PCI Categorical add-in board along with the standard mezzanine card design, Intel has revealed, utilizing a full-height type issue and drawing 600W of energy — making it the go-to selection, the corporate says, for fine-tuning, inference, and retrieval-augmented technology (RAG) workloads.

Naturally, that places it in direct competitors with GPUs and accelerators for NVIDIA — and Intel claims the Gaudi 3 “is projected to ship” a 50 per cent lowering in coaching time for the Llama2-7B and -13B and GPT-3 175B massive language fashions (LLMs), a 50 per cent enhance in inference efficiency, and 40 per cent higher energy effectivity in comparison with NVIDIA’s H100 accelerator.

The brand new accelerator is being made accessible to Intel’s OEM companions, together with Dell, Hewlett Packard Enterprise, Lenovo, and Supermicro first, within the second quarter of this 12 months; common availability will observe within the third quarter for the primary accelerators and within the fourth quarter for the Gaudi 3 PCIe add-in-board variant. Pricing, nonetheless, has but to be disclosed.