# Diffusion Transformers (DiT)


📂 model-architectures

## MODEL ARCHITECTURES

# Diffusion Transformers (DiT)

Use a Transformer over noisy image or video patches as the denoising network inside a diffusion or flow-based generative process.

## MENTAL MODEL

DiT changes the denoiser backbone; it does not eliminate the diffusion process, latent codec, conditioning, or sampler.

## DATA FLOW

- Noisy latent

- Patchify + position / time conditioning

- Transformer blocks

- Predicted latent update

- Sampler step → repeat

## How it trains

The model learns the same class of denoising, velocity, or flow objective as other diffusion systems while Transformer blocks mix information between latent patches.

## How inference runs

A sampler repeatedly invokes the Transformer at different noise or flow times, then a decoder maps the final latent to media.

## Strengths

- •
Transformer scaling and implementation ecosystem

- •
Global interactions between latent patches

- •
Natural path to shared multimodal token processing

## Trade-offs

- •
Attention cost grows with latent token count

- •
Still requires repeated denoiser calls

- •
“DiT” alone does not specify codec, objective, conditioning, or sampler

## Use it when

- Transformer scaling infrastructure is available

- Latent patch count and sampling latency fit the budget

- You need a flexible backbone for image or video generation

## Avoid or challenge it when

- A small U-Net already meets the objective

- The label is being used as a complete architecture specification

- High-resolution token counts overwhelm the target runtime

## Illustrative published families

- • DiT research family

- • Transformer denoisers inside latent image and video pipelines

## Commonly combines with

## PRIMARY REFERENCES

## Model Architectures

## Model Architectures