EdgeCortix Unveils the 60 TOPS SAKURA-II Accelerator, Optimized for On-System Gen AI

Edge machine studying specialist EdgeCortix has introduced the discharge of its next-generation SAKURA-II accelerator — aiming to ship as much as 60 tera-operations per second (TOPS) of energy-efficient compute for on-device massive language fashions (LLMs) and different generative synthetic intelligence (gen AI) workloads.

“SAKURA-II’s spectacular 60 TOPS efficiency inside 8W of typical energy consumption, mixed with its mixed-precision and built-in reminiscence compression capabilities, positions it as a pivotal know-how for the newest generative AI options on the edge,” claims EdgeCortix founder and chief govt officer Sakyasingha Dasgupta

“Whether or not operating conventional AI fashions or the newest Llama 2/3, Steady-diffusion, Whisper, or Imaginative and prescient-transformer fashions,” Dasgupta continues, “SAKURA-II offers deployment flexibility at superior efficiency per watt and cost-efficiency. We’re dedicated to making sure we meet our buyer’s diverse wants and in addition to securing a technological basis that is still sturdy and adaptable inside the swiftly evolving AI sector.”

The SAKURA-II accelerator, which the corporate has been “tailor-made particularly for processing generative AI workloads on the edge,” is able to operating multi-billion parameter AI fashions on-device, together with Llama 2, Steady Diffusion, DETR, and ViT, with a claimed “typical” energy draw of 10W. The chip consists of 20MB of on-device static RAM (SRAM) and delivers its claimed 60 TOPS at INT8 precision, or 30 tera-floating level operations per second (TFLOPS) at BF16.

For these working with space-constrained gadgets, the SAKURA-II is being made obtainable on an M.2 2280-footprint PCI Specific module; for workstations and servers, a full-size PCI Specific add-in board (AIB) variant hosts one or two SAKURA-II chips to ship as much as 120 TOPS per card. The M.2 variant is out there with 8GB or 16GB of LPDDR4 reminiscence, whereas the PCIe AIB is out there with 16GB in single-chip or 32GB in dual-chip variants — with the latter, naturally sufficient, doubling the everyday energy draw to 20W.

The accelerator is backed by EdgeCortix’s MERA software program stack, which it says delivers assist for a variety of fashions together with conventional convolutional neural networks (CNNs) like ResNet 50/101 and YoloX and transformer-based fashions together with DINO, GPT-2, Open-Llama2, and Llama 3 — the latter operating on-device at an eight-billion parameter dimension.

The SAKURA-II playing cards are actually obtainable to pre-order forward of a deliberate launch within the second half of the 12 months, priced at $249 for the M.2 8GB, $299 for the M.2 16GB, $429 for the single-chip 16GB PCIe AIB, and $749 for the double-chip 32GB PCIe AIB. Whereas EdgeCortix had confirmed plans to additionally promote the SAKURA-II as a standalone chip for these seeking to combine it into their very own gadget designs, it had not launched pricing on the time of writing.