Concept card

Encoder-Decoder vs Decoder-Only — Why GPT/Claude/Gemini All Chose Decoder-Only

As of 2026-07-24T14:00:00Z

Edited by Cédric-Olivier Mâry, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is Encoder-Decoder vs Decoder-Only — Why GPT/Claude/Gemini All Chose Decoder-Only?

Almost every frontier LLM converged on decoder-only architecture within five years of the original encoder-decoder Transformer because a single stack halves engineering complexity and makes in-context learning fall out naturally from treating prompt and generation as one sequence.

The original 2017 "Attention Is All You Need" Transformer was an encoder-decoder architecture: an encoder stack that reads the entire input bidirectionally, paired with a decoder stack that generates output autoregressively while cross-attending to the encoder's representations. Within about five years, nearly every frontier large language model — GPT-3 and GPT-4, Claude, Llama, Gemini, Mistral, DeepSeek — had converged on a different design: decoder-only, a single stack that processes both the input and the generated output as one continuous sequence under causal masking, with no separate encoder and no cross-attention. Understanding why that convergence happened, and where it did not happen, is essential for anyone evaluating model architectures, scoping a fine-tuning project, or explaining to a client why "the chatbot" and "the translation engine" are quietly built on different foundations.

Three families, not two

Encoder-only models — BERT, RoBERTa, DeBERTa — use bidirectional attention and a masked-language-model training objective; they are excellent at classification, extractive question answering and producing embeddings, but they were never designed to generate fluent free text and essentially cannot. Encoder-decoder models — T5, BART, the original Transformer, mT5 — pair a bidirectional encoder with a causal decoder connected by cross-attention, and remain the natural fit for tasks that transform one structured sequence into another: translation, summarization, and structured-input-to-structured-output problems. Decoder-only models — the GPT family, Claude, Llama, Mistral, Gemini, DeepSeek — use causal attention throughout and a next-token-prediction objective, optimized for open-ended generation and for learning new tasks from examples placed directly in the prompt.

Why decoder-only won

Why it matters

Encoder-only models (BERT-class) can't generate fluent text at all — they're built for classification, extractive QA and embeddings only.
Causal masking in decoder-only models enables KV-cache reuse across generation steps in a way encoder-decoder cross-attention complicates.
Encoder-decoder architectures still win for pure translation and fixed-schema structured-to-structured transformations — decoder-only isn't universally better, just better for general-purpose generation.

Key points

Three families — encoder-only (BERT, classification/embeddings), encoder-decoder (T5/BART, structured transforms), decoder-only (GPT/Claude/Llama/Gemini, open generation).
Decoder-only won at frontier scale — one stack simpler than two; in-context learning emerges naturally; KV cache reuse cleaner; full param budget for both understanding and generation.
Encoder-decoder still wins — production translation (Google), fixed-schema structured transforms, long-input/short-output summarisation.
Practical default for new LLM projects in 2026 — decoder-only; the ecosystem (Llama 3, Mistral, DeepSeek, Claude, GPT, Gemini) is decoder-only and tooling is decoder-only-first.
Encoder-only (BERT-family) still dominant for embeddings + classification — different use case, not a competitor to LLM generation.
Encoder-Decoder vs Decoder-Only — Why GPT/Claude/Gemini All Chose Decoder-Only is mastered only when it changes a named buyer decision.
Start with the semantic contract and control model before demonstrating the tool.
Use current SAP, analyst, study, KG, and news signals as evidence, not decoration.
Separate verified facts from directional trends and modeled assumptions.
Define owner, metric, threshold, support path, and rollback before scaling.

Terms used on this page

Decoder-only transformer: Transformer architecture with a single stack of layers, causal self-attention throughout, trained on next-token prediction; the canonical architecture of GPT, Claude, Llama, Mistral, Gemini, and DeepSeek.
Encoder-decoder transformer: The original 2017 transformer architecture: a bidirectional encoder reads the full input, a causal decoder generates output while cross-attending to encoder representations; canonical for T5, BART, and production translation systems.
Causal masking: An attention mask that prevents each token from attending to future tokens in the sequence; required for autoregressive generation and the defining feature of decoder-only architectures.
Cross-attention: Attention from decoder tokens to encoder representations in encoder-decoder transformers; the mechanism by which the decoder consults the input. Absent in decoder-only models.
In-context learning (ICL): The ability of an LLM to perform a new task from examples provided in the prompt, without any parameter update; emerges naturally in decoder-only models trained on next-token prediction at scale.
Decision owner: The accountable person who accepts the trade-off and funds the next action.
Semantic contract: The shared definition of business terms, metrics, entities, and access rules used by tools and teams.
Control plane: The layer that applies policy, access, lineage, monitoring, and escalation across the operating model.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks · the facts worth quoting.

Open in the app →