Diffusion & score-based generation

📂 model-architectures

MODEL ARCHITECTURES#

Diffusion & score-based generation

Learn to reverse a gradual noising process, producing data by repeatedly transforming noise toward a sample.

MENTAL MODEL#

Start with static and repeatedly remove the predicted noise, guided by a time step and optional condition.

DATA FLOW#

• Clean training sample + sampled noise level

• Noisy sample

• U-Net or Transformer denoiser

• Noise / velocity / score estimate

• Iterative reverse sampler

How it trains#

A network predicts noise, clean data, velocity, or a score at randomly sampled noise levels. Related flow-matching formulations learn a vector field between distributions.

How inference runs#

Sampling integrates a learned reverse process across multiple steps. Sampler choice, step count, guidance, and seed trade speed, diversity, and fidelity.

Strengths#

• • Stable training relative to adversarial objectives

• • Flexible conditioning and editing

• • Strong coverage across image, audio, video, and scientific data

Trade-offs#

• • Iterative sampling is slower than a one-pass generator

• • Guidance can reduce diversity or introduce artifacts

• • The data representation and sampler materially affect results

Use it when#

• High-dimensional conditional generation is central

• Multiple inference steps fit the latency budget

• Controls and sample quality can be evaluated jointly

Avoid or challenge it when#

• One-pass generation is mandatory

• The model must run on a severely constrained device

• “Diffusion” is being treated as one fixed implementation

Illustrative published families#

• • DDPM

• • Score-based models

• • Pixel-space diffusion systems

Commonly combines with#

PRIMARY REFERENCES#

Model Architectures#

Model Architectures#