Just a day after Mistral announced its best open-source language model Mixtral 8x22B, Meta is here to challenge its lead with Llama 3, which is already powering its Meta AI. Llama 3 is not only the new top open-source model but even rivals proprietary ones such as Google’s flagship Gemini 1.5 Pro.

For now, Meta has only unveiled the first two models in its new Llama 3 generation of language models, 8B and 70B, which, as the names say, are trained on 8 billion and 70 billion parameters. Larger Llama 3 AI models will likely be unveiled later this year.

Quite like Mistral’s new Mixtral 8x22B, Llama 3 also specializes in contextual understanding and complex tasks like translation and dialogue generation. Additionally, Meta claims that Llama 3 represents a big leap in reasoning, coding, and instruction-following skills.

Llama 3’s landing page says: “With enhanced scalability and performance, Llama 3 can handle multi-step tasks effortlessly, while our refined post-training processes significantly lower false refusal rates, improve response alignment, and boost diversity in model answers.”

The landing page also sheds light on the hardware used to power Llama 3. The new open-source AI model has been trained on a cluster of 24,000 custom-built GPUs which had over 15 trillion tokens of data. This training dataset is 7x larger than the one used for Llama 2, including 4x more code. The context length has also been doubled to reach 8K and the knowledge cut-off date is March 2023 for Llama 8B and December 2023 for Llama 80B.

When it comes to industry standard benchmark results, Llama 3 not only beats Gemini 1.5 Pro but also Anthropic’s Claude 3 Sonnet, but does not match up to OpenAI’s GPT 4 Turbo, or Anthropic’s most powerful Claude Opus.

This jump in performance can be credited to the whopping 15 trillion aforementioned training data, which has all been sourced from public platforms, at least according to Meta. This data impressively covers 30 languages and over 5% of the total data is not in English. However, Meta has said that it does not expect Llama 3 to have the same performance as English in other languages.

As mentioned earlier, Llama 3 is already powering Meta AI, which has now made its way to WhatsApp as well as Instagram in a limited test.