Highlights:

  • The company has created an open-source tool named Hidet, which can lower the expenses associated with operating AI models and enhance their inference capabilities.
  • CentML asserts that its software accelerates AI models without compromising their precision or necessitating manual code adjustments by developers.

Recently, CentML Inc., a startup focused on enhancing AI model performance, announced that it has successfully completed a seed round of funding totaling USD 27 million.

The Gradient Ventures startup fund of Alphabet Inc. led the investment. Additional contributions came from Nvidia Corp.,  Thomson Reuters Ventures, Deloitte Ventures, and Radical Ventures.

CentML was established in 2022 by AI researchers affiliated with the University of Toronto. The company has developed an open-source tool called Hidet to reduce the cost of running AI models and improve their inference performance. Additionally, CentML manages another open-source project to accelerate AI training.

A compiler or tool called Hidet enables developers to transform raw source code into a functional program, specifically focusing on optimizing neural networks’ source code. During the assembly of an AI model, Hidet performs optimizations to enhance the model’s inference performance.

The tool accelerates neural networks by employing a technique known as operator fusion to optimize their code. Operators within an AI model perform various functions, including selecting artificial neurons for specific computing tasks. Hidet has the capability to consolidate multiple operators into a single code component, enhancing performance by minimizing the need for data transfer between memory.

The AI the tool compiles is packaged into what are known as CUDA kernels. Those are pieces of code designed to function best on graphics processing units made by Nvidia Corp. A CUDA kernel’s main advantage is its ability to be split among numerous GPU cores, which boosts efficiency.

According to CentML, its software accelerates AI models without lowering their accuracy or necessitating manual code modifications from developers. The business tripled the speed of the open-source Llama 2 model from Meta Platform Inc. in an internal project. According to CentML, some users have seen even greater gains in performance.

Accelerating an AI model lowers infrastructure costs in addition to processing times. When a neural network’s performance doubles, it can process data at the same rate as before while utilizing only half the GPUs. Consequently, organizations need to procure fewer chips for their machine-learning initiatives.

In addition to Hidet, CentML provides DeepView, another open-source tool. It enables developers to keep an eye on the training process of an AI model and identify methods to expedite it. DeepView aims to maximize neural networks constructed using PyTorch, a well-liked AI development framework.

Vinod Grover, Director of CUDA and compiler software at Nvidia, said, “The proliferation of generative AI is creating a new base of developers, researchers, and scientists seeking to use accelerated computing for a host of capabilities. CentML’s work to optimize AI and ML models on GPUs in the most efficient way possible is helping to create a faster, easier experience for these individuals.”

The money raised from CentML’s seed round is apparently going to be used for research and product development projects. The organization currently employs thirty people and plans to add more workers to help with its expansion.