Meta presents Llama 4: the new generation of its multimodal AI now integrates Expert Mixture architecture

Meta has expanded its family of large-scale language models (LLM) with the introduction of Llama 4. The technology company has developed three versions of this multimodal AI (Scout, Maverick, and Behemoth), with the aim of assisting developers and professionals in creating more personalized experiences.

One noteworthy aspect of the Llama 4 models is that they are Meta’s first models to utilize a Mixture of Experts (MoE) architecture. This means that, instead of having a single neural network that processes all the information, the AI possesses an advanced neural network composed of several “experts,” which are subnetworks or models specialized in specific tasks.

Thanks to this, only the necessary “experts” will be activated for each query, improving efficiency and reducing latency or response time. Other AIs that are based on MoE include DeepSeek V3, Qwen2.5-Max, or Gemini 1.5 Pro.

The Llama 4 Scout and Llama 4 Maverick models are now downloadable from llama.com and Hugging Face. Additionally, the company has stated that, in the coming days, they will also be available through their partners. Moreover, the technology of Llama 4 has been activated in Meta AI, both on its website and on WhatsApp, Messenger, and Instagram.

However, Meta makes it very clear in its usage policy that the rights for use and distribution of Llama 4 “are not granted if you are an individual or a company headquartered in the European Union. This restriction is not applicable to end users of a product or service incorporating such multimodal models.”

On April 29, during its developer event, LlamaCon, the technology giant is scheduled to share more information about Llama 4.

Today is the start of a new era of natively multimodal AI innovation.

Today, we’re introducing the first Llama 4 models: Llama 4 Scout and Llama 4 Maverick — our most advanced models yet and the best in their class for multimodality.

Llama 4 Scout
• 17B-active-parameter model… pic.twitter.com/Z8P3h0MA1P

— AI at Meta (@AIatMeta) April 5, 2025

Llama 4 Models

The Llama 4 models have been designed with native multimodality, being able to understand and generate text, images, and even video as part of its core functionality, rather than as an added capability.

Additionally, it incorporates “early fusion”, a technique that allows for the combination of different data (whether text, images, or video), from the initial layers of the model, instead of processing them separately. “Early fusion represents a significant breakthrough, as it allows us to jointly pre-train the model with large amounts of unlabeled text, image, and video data“, explains Meta.

The technology company has also improved the vision encoder in Llama 4, which is based on MetaCLIP but has been adapted with another Llama model for improved performance.

Llama 4 Scout

This is the more compact version of Llama 4, yet it is still more powerful than all previous-generation Llama models from previous generations. It has been designed to operate on a single GPU and, according to Meta, has outperformed other models such as Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1 in various tests.

Llama 4 Scout is a model with 17 billion active parameters and 16 “experts” offering a context window of 10 million tokens.

Remember that parameters are the internal values that a model learns during training. They are not the data set itself, but rather the controls and instructions built into its system allowing it to define how to process and transform the information it analyzes.

Llama 4 Maverick

Meanwhile, Llama 4 Maverick is a model with 17 billion active parameters and 128 “experts”. Meta claims that “it is the best multimodal model in its class, surpassing GPT-4o and Gemini 2.0 Flash across a wide range of widely publicized benchmarks, while achieving results comparable to the new DeepSeek v3 in reasoning and coding, with less than half the active parameters.”

This model also stands out for its enhanced image and text comprehension capabilities. It also offers a great cost-performance ratio, with an experimental chat version scoring an ELO of 1,417 in LMArena.

Llama 4 Behemoth

This is the model that has served as a guide for the creation of Scout and Maverick, and which Meta claims will continue to do so for future versions. The company describes it as “one of the world’s most intelligent LLM models and the most powerful to date.”

For the moment, Llama 4 Behemoth is still under development, but it already yields results surpassing GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on STEM-focused benchmarks. It has 288 billion active parameters and includes 16 “experts” and nearly 2 trillion total parameters.

Photo: GPT-4o