# Decoder-only Transformers


📂 model-architectures

## MODEL ARCHITECTURES

# Decoder-only Transformers

Use a causal mask so each position sees only earlier tokens, matching the left-to-right generation process used by most general-purpose chat models.

## MENTAL MODEL

Compress the prefix into attention state, predict one continuation token, append it, and repeat.

## DATA FLOW

- Prefix tokens

- Causally masked self-attention

- Hidden state at each position

- Vocabulary logits

- Next token → repeat

## How it trains

Next-token cross-entropy supplies a learning signal at every position. Post-training may add instruction examples, preferences, verifiable rewards, or tool trajectories.

## How inference runs

Prefill processes the prompt; decode then emits tokens serially while the KV cache stores past attention keys and values. Context size, cache memory, and output length drive serving behavior.

## Strengths

- •
One scalable interface for many generative tasks

- •
Strong in-context adaptation

- •
Can interleave prose, code, structured data, and tool-call tokens

## Trade-offs

- •
Serial decode latency

- •
Quadratic prefill in standard full attention

- •
Prompt sensitivity and probabilistic output require validation

## Use it when

- Open-ended generation or interactive chat is central

- Many tasks can share one model endpoint

- You can constrain and evaluate outputs in the product loop

## Avoid or challenge it when

- A small encoder or deterministic component meets the requirement

- Guaranteed exact outputs are assumed without validation

- Long source corpora should be searched rather than placed wholesale in context

## Illustrative published families

- • GPT-style causal language models

- • LLaMA research family

## Commonly combines with

## PRIMARY REFERENCES

## Model Architectures

## Model Architectures