Highlights:

  • RNGD is built on TCP architecture, offering an optimal balance of performance, programmability, and efficiency.
  • The RNGD chip features a 1 GHz frequency, 1.5 MB memory bandwidth, 256 MB on-chip RAM, and a PCIe Gen5 x16 interconnect supporting up to 64 Gbps throughput.

A semiconductor startup, FuriosaAI Inc., unveiled a new accelerator chip for multimodal AI and large language models. The company sharply emphasizes artificial intelligence.

Recently, during the Hot Chips 2024 conference at Stanford University, the company introduced its newest chip, which goes by the moniker RNGD, or “Renegade.” Users will be able to use it later in the year, but for now, early-access clients can sample it.

Furiosa claims that the RNGD chip is a highly effective data center accelerator intended to facilitate multimodal model inference and high-performance LLMs. The firm positions it as a substitute for graphics processing units made by Nvidia Corp.

According to the business, the Tensor Contraction Processor, or TCP architecture, on which RNGD is built offers the ideal compromise between performance, programmability, and efficiency. With a Thermal Design Power of 150 watts, it has some impressive specs compared to some of the top GPUs available today, which have a TDP of over 1,000 watts.

Moreover, Furiosa promises exceptionally high performance as the CPU has 48 gigabytes of high-bandwidth memory. This enables the effective use of a single card for open-source LLMs like Llama 3.1 8B from Meta Platforms Inc.

The RNGD chip has a frequency of one gigahertz, 1.5 megabytes of memory bandwidth, 256 megabytes of on-chip standard random-access memory, and a PCIe Gen5 x16 interconnect that can support throughput of up to 64 gigabits per second. It was created by five-nanometer process of the Taiwan Semiconductor Manufacturing Co.

A robust compiler, which treats entire AI models as a single-fused process and is co-designed to be optimized for TCP-based devices, makes programmability possible. The company implies that the RNGD chips can be customized to run almost any LLM or multimodal AI application.

These figures indicate that the Furiosa RNGD chip can run some of the most well-known LLMs with remarkable capability. In fact, the startup’s models, like OpenAI’s GPT-J 6B, have shown outstanding performance on industry-standard benchmarks, with the model processing 15.13 queries per second.

Furiosa’s lineage is respectable. Three hardware and software engineers who had previously worked for major chip manufacturers like Advanced Micro Devices Inc., Qualcomm Inc., and Samsung Electronics Co. Ltd. formed it in 2017.

Since its beginning, the company has prioritized quick product delivery and iteration. In the ResNet-50 image classification and SSD—MobileNetV1 object identification benchmarks, its first-generation chip, dubbed Warboy, performs favorably with some of Nvidia’s older GPU designs. It is a high-performance data center accelerator tailored for computer vision tasks.

Furiosa Co-founder and Chief Executive June Paik mentioned that RNGD is the outcome of years of innovation the startup spent. “RNGD is a sustainable and accessible AI computing solution that meets the industry’s real-world needs for inference. With our hardware now running LLMs at full speed, we’re entering an exciting phase of continuous advancement,” he added.