Highlights:

  • Code Llama 70B, derived from Meta’s open-source Llama 2 large language model, is tailored for generating code from natural language prompts.
  • Meta stated that Code Llama 70B underwent training on over 500 billion tokens comprising code and associated data.

Meta Platforms Inc. has just unveiled an upgraded edition of its code-generating artificial intelligence model, Code Llama. This enhanced version boasts improved processing capabilities and higher accuracy, as well as added support for various programming languages, including Python, C , Java, and PHP.

Code Llama 70B is a variant of Meta’s open-source Llama 2 large language model, tailored for generating code from natural language prompts. Generating code is frequently cited as one of the most promising applications for generative AI. Proponents argue that by streamlining code production, generative AI can enhance the efficiency and accessibility of software development, empowering developers to explore more innovative solutions.

The issue with code generation is that it has to be incredibly exact and inflexible, adhering to tight guidelines and syntax, and yet predictable and functioning as the user would expect. However, software code is frequently lengthy and intricate, demanding extensive contextual understanding and logic from AI models to comprehend and generate it effectively. Achieving this level of proficiency necessitates substantial computing power, and this is where Code Llama 70B excels.

Code Llama 70B, being one of the largest open-source code generation models to date, stands poised to emerge as a significant benchmark within the realm of AI focused on generating code. Its potential is considerable, as numerous prior attempts at code generation AI have faced challenges in reliably and securely automating software development processes.

Meta stated that Code Llama 70B underwent training on over 500 billion tokens comprising code and associated data, indicating a substantial enhancement in capability and robustness compared to its predecessors. Additionally, with a widened context window of 100,000 tokens, it gains the ability to generate more intricate code structures.

Holger Mueller from Constellation Research Inc. informed a leading media house that code-generating models hold particular intrigue due to the prevalent challenge many enterprises face in sourcing an adequate number of developers to construct the required software. “Meta’s Code Llama 70B promises to make the idea of autonomous software operations or ASO a reality, leading us into a world where software writes software. It makes sense for Meta to pursue this idea as it needs to create a lot of code for its own projects, and Code Llama brings us closer to the day where 100% of enterprise-ready software may be written by generative AI,” the analyst said.

Chief Executive of Meta, Mark Zuckerberg, announced the new model personally in a public post on Facebook. He said, “Writing and editing code has emerged as one of the most important uses of AI models today. The ability to code has also proven to be important for AI models to process information in other domains more rigorously and logically. I’m proud of the progress here, and looking forward to including these advances in Llama 3 and future models as well.”

The company is releasing multiple versions of Code Llama 70B to the public, among them CodeLlama-70B-Instruct, which are tailored to process code requests in natural language. In parallel, CodeLlama-70B-Python has been optimized specifically for generating code in Python language, bolstered by an extra 100 billion tokens derived from Python code to enhance its accuracy and fluency in the language.

The Code Llama 70B models are currently accessible for download through a variety of platforms, including Hugging Face, PyTorch, Jupyter Notebook, and TensorFlow.