# Latent diffusion


📂 model-architectures

## MODEL ARCHITECTURES

# Latent diffusion

Run diffusion in an autoencoder’s lower-dimensional latent space, then decode the generated representation back to pixels or another signal.

## MENTAL MODEL

Denoise a compact learned sketch instead of every raw pixel.

## DATA FLOW

- Training media

- Frozen or jointly trained autoencoder latent

- Conditional latent denoiser

- Sampled clean latent

- Decoder → media

## How it trains

An autoencoder first learns a perceptual compression. The diffusion model then learns the denoising objective over those latents, often conditioned through cross-attention.

## How inference runs

Sample in the smaller latent tensor over several steps and decode once. Image-to-image and inpainting can begin from encoded and selectively noised source content.

## Strengths

- •
Lower denoising cost than raw high-resolution pixels

- •
Modular text conditioning and editing

- •
Reusable autoencoder and generator components

## Trade-offs

- •
The autoencoder creates a reconstruction ceiling

- •
Fine text or high-frequency detail can be lost in compression

- •
Two-stage failures are harder to attribute

## Use it when

- High-resolution media generation must fit practical compute

- Editing and conditional control are required

- Codec reconstruction is validated on the target domain

## Avoid or challenge it when

- The autoencoder drops task-critical details

- End-to-end simplicity matters more than generation cost

- Latent-space shortcuts could hide safety-relevant content

## Illustrative published families

- • Latent Diffusion Models

- • Stable Diffusion research lineage

## Commonly combines with

## PRIMARY REFERENCES

## Model Architectures

## Model Architectures