choosing-a-base-model
Choosing a Base Model
📂 fine-tuning
FINE-TUNING GUIDE#
Choosing a Base Model
Select from evidence on your workload, not a generic leaderboard. Start with the smallest checkpoint that meets your quality target, then verify its terms, training support, and deployment path before investing in a full run.
Run a base-model bake-off first#
1. Freeze the test set#
Build representative success, failure, boundary, safety, and out-of-distribution cases. Keep them out of training and prompt iteration.
2. Compare baselines#
Test the same rubric with prompting, few-shot examples, retrieval, and candidate base models. Fine-tuning should solve a measured gap.
3. Pilot cheaply#
Run a short adapter experiment, record the full configuration, and promote only if the gain survives held-out and safety evaluation.
Selection gate#
-
The untuned checkpoint beats a prompt-only and retrieval baseline on your held-out cases.
-
The license and acceptable-use terms cover your product, geography, distribution, and derivatives.
-
The tokenizer, chat template, context length, and output format match the intended serving stack.
-
A small pilot fits with enough headroom for activations, optimizer state, batches, and checkpoints.
-
The serving stack supports the adapter or merged checkpoint format you intend to ship.
-
You can test task quality, safety regressions, latency, throughput, and cost before promotion.
Illustrative open-weight families#
Snapshot reviewed August 2026. This is a starting set, not a ranking or a promise that every checkpoint supports every tuning method. Follow the official source for current files and terms.
Gemma#
A Google-supported open-weight family with documented local and cloud tuning paths.
Check: Choose the exact checkpoint and format only after checking the current Gemma terms, framework guide, and memory requirements.
Qwen#
A broad open-weight family worth evaluating for multilingual, coding, and multimodal workloads.
Check: The collection contains models with very different architectures and sizes; verify each model card and chat template.
Mistral#
Open-weight models with official deployment and, for selected checkpoints, fine-tuning guidance.
Check: Some recent checkpoints are mixture-of-experts models whose total weight footprint is much larger than their active parameter count.
gpt-oss#
OpenAI open-weight reasoning models with an official model card and fine-tuning notes.
Check: Preserve the required Harmony response format and benchmark memory on the exact training stack before selecting a checkpoint.
Nemotron#
NVIDIA’s open-weight family aimed at agent and tool-use workloads, in tiers from compact to very large mixture-of-experts checkpoints, with data and recipes published for parts of the family.
Check: Tiers differ in license and architecture; verify the full weight footprint of a mixture-of-experts checkpoint and your framework’s support for it before piloting.
GLiNER#
Small encoder models for extraction, classification, and PII detection that train quickly and serve cheaply; tuned task models of this class can match far larger generalists on narrow tasks.
Check: These are schema-driven task models, not chat models; confirm the task fits the interface and beat a generalist baseline on your own labels before committing.
Measure memory#
Parameter count alone does not predict training memory. Precision, sequence length, attention implementation, optimizer, batch size, adapters, and sharding all matter. The costliest boundary is the single accelerator: once a pilot no longer fits one device even with adapters and quantization, training becomes a distributed-systems project.
Read the exact license#
“Open weight” is not a license category: some releases publish weights, data, and training code under permissive terms, while others ship weights alone with revenue or field-of-use conditions. Review the checkpoint’s terms and notices; do not infer rights from the model family name.
Re-run safety tests#
A tuned model can regress outside the target task. Test base capabilities, abuse cases, privacy leakage, and refusal behavior before each release.
Never purchase hardware from a generic VRAM estimate. Reproduce a representative pilot with the exact checkpoint, precision, sequence length, and framework, or obtain a current provider quote that includes training, storage, serving, and data-transfer costs.