embedding-models
Embedding models
📂 model-architectures
MODEL ARCHITECTURES#
Embedding models
Map text, images, audio, users, or items into vectors whose geometry is trained to preserve a useful notion of similarity.
MENTAL MODEL#
A learned coordinate system. Nearness means “similar under the training objective,” not universally equivalent or factually related.
DATA FLOW#
-
Input item
-
Encoder
-
Pooling or projection
-
Fixed-length vector
-
Similarity search, clustering, or classifier
How it trains#
Contrastive, metric-learning, classification, or paired-data objectives pull useful matches together and push negatives apart. The negative-sampling strategy and domain data define what similarity means.
How inference runs#
Each item is encoded in one forward pass. Stored vectors enable fast approximate-nearest-neighbor search; a cross-encoder can rerank the small candidate set for finer interaction.
Strengths#
-
• Efficient semantic retrieval over large collections
-
• Reusable features for clustering, routing, recommendations, and deduplication
-
• Candidate vectors can be computed and indexed ahead of time
Trade-offs#
-
• A single vector compresses away token-level detail
-
• Similarity degrades under domain, language, or time drift
-
• Scores are model- and index-specific, not calibrated probabilities
Use it when#
-
You need semantic candidate retrieval
-
The corpus is too large for pairwise scoring
-
You can evaluate recall on production-shaped queries
Avoid or challenge it when#
-
Exact lexical matching is the only requirement
-
You need a generated response rather than a representation
-
A similarity threshold would be deployed without calibration
Illustrative published families#
-
• BERT-derived sentence encoders
-
• The separate text and image towers used by CLIP