Highlights:

  • GroqCloud provides developers with on-demand access to LPUs for their AI applications, allowing them to get acquainted with the company’s chips and optimize their designs for the architecture.
  • Groq develops AI chips and software to supercharge AI applications, aiming to compete with industry leaders like Nvidia.

Recently, a startup specializing in artificial intelligence and machine learning chip manufacturing, Groq Inc., has secured USD 640 million in a late-stage funding round led by Blackrock Inc.

The startup designs semiconductor chips and software to optimize deployed AI activities, known as inference, aiming to compete with industry giants like Nvidia Corp. The Series D funding round values the company at USD 2.8 billion and increases the total raised to date to over USD 1 billion, including a USD 300 million Series C round in 2021.

The Series D funding round drew investments from both new and existing backers, including Neuberger Berman, Type One Ventures, Cisco Investments, Global Brain’s KDDI Open Innovation Fund III, and Samsung Catalyst Fund.

The company was founded in 2016 by Chief Executive Jonathan Ross, a former Google LLC engineer who created the search giant’s TPU machine learning processors. Its flagship product, the LPU Inference Engine, is an AI chip designed to power large language models in production after they have been developed and trained. LPU stands for Language Processing Unit.

In a speed test conducted in November, Groq set an inference speed record with Meta Platforms Inc.’s Llama 2 70B large language model. The company’s chips and software stack achieved a new benchmark for performance and accuracy, processing more than 300 tokens per second per user with the Meta AI model.

Since then, the company has upgraded its stack to support Meta’s largest open model, Llama 3.1 405B, on its hardware. This update also includes compatibility with other Llama 3.1 models in the family, such as 70B Instruct, 40B Instruct, and 8B Instruct.

“You can’t power AI without inference compute. We intend to make the resources available so that anyone can create cutting-edge AI products, not just the largest tech companies…. Training AI models is solved, now it’s time to deploy these models so the world can use them,” said Ross.

Ross mentioned that the new funding will allow the company to add more than 100,000 LPUs to GroqCloud, its cloud-based AI inference service. Developers can use this service to swiftly and easily build and deploy AI applications utilizing popular industry large language models (LLMs) such as Meta’s Llama 3.1, OpenAI’s Whisper Large V3, Google’s Gemma, and Mistral AI’s Mixtral.

GroqCloud provides developers with on-demand access to LPUs for their AI applications, enabling them to get acquainted with the company’s chips and optimize their use of the architecture. Groq developed this cloud service with the assistance of Definitive Intelligence, a Palo Alto, California-based analytics provider that the company acquired in March.

“Having secured twice the funding sought, we now plan to significantly expand our talent density. We’re the team enabling hundreds of thousands of developers to build on open models and — we’re hiring,” Ross added.