# Fine-Tuning Techniques


📂 fine-tuning

## FINE-TUNING GUIDE

# Fine-Tuning Techniques

Make two separate choices: what learning objective matches your data, and how much of the model to update. No method wins across every model, task, and hardware stack.

## 1. Choose the learning objective

### Supervised fine-tuning (SFT)

Data: Input and desired-output examples

Teach a stable task, response structure, style, or tool-calling pattern.

Watch: The model may imitate errors and biases in demonstrations; unrelated behavior can regress.

### Preference optimization

Data: Preferred and non-preferred responses, or equivalent feedback

Shift choices where quality is easier to compare than to specify as one ideal answer.

Watch: Label policy, annotator agreement, and held-out evaluation matter as much as the algorithm.

### Reinforcement fine-tuning

Data: Prompts plus a reliable grader or reward signal

Optimize outcomes that can be scored consistently across generated trajectories.

Watch: Reward hacking, grader bias, instability, and higher operational complexity require extra controls.

## 2. Choose the parameter-update method

### Full-parameter tuning

Updates the base weights and offers the most flexibility, with the largest memory, storage, and rollback burden.

Consider only when adapter methods miss a measured target and the infrastructure and data justify it.

### LoRA / adapters

Freezes the base model and trains smaller update modules, often under one percent of the parameter count. Artifacts are compact and multiple adapters can share one base.

A sensible first experiment for many transformer workloads; rank and target modules still need empirical tuning.

### QLoRA

Backpropagates through a quantized frozen base into LoRA adapters, reducing base-weight memory at potential compatibility and throughput costs.

Useful when memory is the binding constraint; validate quantization support and final serving quality.

## Variants are hypotheses, not upgrades

### Adapter variants

DoRA, AdaLoRA, LoRA+, and other variants change parameterization, allocation, or optimization. Try one only when its implementation supports your model and it addresses a measured limitation of the simpler baseline.

### Quantization-aware training

QAT targets quality after low-precision deployment. It is different from QLoRA, which primarily uses a quantized frozen base to reduce training memory. Validate the exported format on the intended runtime.

## Primary references

## Fine-Tuning Guide

## Fine-Tuning Guide