contrastive-dual-encoders
Contrastive & dual encoders
📂 model-architectures
MODEL ARCHITECTURES#
Contrastive & dual encoders
Encode two inputs independently (such as a query and document or image and caption) and train matching pairs to land near each other.
MENTAL MODEL#
Two readers meet in a shared coordinate system; they are fast because candidates do not interact until their vectors are compared.
DATA FLOW#
-
Paired inputs A and B
-
Separate or shared encoders
-
Projected normalized vectors
-
Similarity matrix
-
Contrastive matching loss
How it trains#
Batch contrastive objectives reward paired examples and treat other examples as negatives. Negative quality, duplicate semantics, and temperature materially affect the learned space.
How inference runs#
Encode each side independently and compare vectors with dot product or cosine similarity. Precompute the large candidate side; optionally rerank top results with a cross-encoder.
Strengths#
-
• Scales retrieval to large corpora
-
• Aligns modalities without a joint decoder
-
• Enables zero-shot classification through label text in some settings
Trade-offs#
-
• Independent encoding misses fine-grained cross-input interactions
-
• False negatives can distort training
-
• Global similarity may ignore spatial, temporal, or compositional details
Use it when#
-
Fast retrieval or matching is the first stage
-
One side can be indexed offline
-
Recall is followed by task-appropriate reranking when needed
Avoid or challenge it when#
-
Every candidate needs deep token-to-token comparison
-
The task depends on precise spatial relationships
-
Raw similarity scores are assumed to be probabilities
Illustrative published families#
-
• CLIP image–text dual encoder
-
• Dense passage retrieval
-
• Bi-encoder semantic search