# Autoencoders, VAEs & learned tokenizers


📂 model-architectures

## MODEL ARCHITECTURES

# Autoencoders, VAEs & learned tokenizers

Learn an encoder that compresses data and a decoder that reconstructs it; variational and quantized variants shape the latent space for sampling or downstream generation.

## MENTAL MODEL

A learned codec. It decides which information survives before another model generates in the compressed space.

## DATA FLOW

- Image, audio, or other signal

- Encoder

- Continuous latent or discrete codes

- Optional latent generator

- Decoder reconstruction

## How it trains

A reconstruction objective preserves perceptual content. VAEs add a distribution regularizer; vector-quantized models map representations to a discrete codebook and add codebook/commitment objectives.

## How inference runs

Encode for compression or editing; sample or generate a latent/code sequence; decode to the original modality. Reconstruction quality limits the best possible downstream output.

## Strengths

- •
Reduces the dimensionality of expensive media

- •
Creates reusable continuous latents or discrete tokens

- •
Separates representation learning from the generative prior

## Trade-offs

- •
Compression discards detail

- •
Latent geometry may not match downstream semantics

- •
Quantized codebooks can be underused or introduce artifacts

## Use it when

- Raw media is too expensive to model directly

- A generative model needs a compact continuous or discrete space

- Reconstruction failures are measured by content slice

## Avoid or challenge it when

- Lossless reconstruction is mandatory

- The codec domain differs materially from production data

- Decoder artifacts would be mistaken for generator failures

## Illustrative published families

- • Variational Autoencoder (VAE)

- • VQ-VAE discrete visual/audio tokenizers

- • Autoencoders used by latent diffusion

## Commonly combines with

## PRIMARY REFERENCES

## Model Architectures

## Model Architectures