For ~80% of LLM use cases, prompt engineering + RAG is sufficient. For ~15%, better model routing or a larger model solves the problem. Only ~5% truly require fine-tuning — usually domain-specific tasks, format control beyond what prompting achieves, or latency/cost requirements that demand a smaller, specialized model.

Build your golden test set before writing a single training example. If you can't measure improvement, you're not doing engineering — you're hoping. The eval set defines what "better" means for your specific use case.

Choose the smallest model that already performs reasonably well on your task with prompting alone. Fine-tune to specialize and improve consistency — not to compensate for fundamental capability gaps. A 7B model that handles your domain well will outperform a poorly-matched 70B model.

Fine-tuning makes economic sense when: (1) you've proven prompting doesn't work, (2) you have 5K–50K+ quality training examples, (3) you need the specialized capability for high-volume production, and (4) you're prepared to maintain the fine-tuned model over time (updates when the base model changes, dataset maintenance, eval pipeline). If any of these are missing, the ROI is negative.

Compute cost for a single training run is small. The real cost is iteration time: reviewing failures, curating fixes, retraining, re-evaluating. Budget for 3–10 iteration cycles. A team that plans for one training run will be surprised; a team that plans for ten will ship a good model.

Before fine-tuning, explicitly document: (1) What tasks must improve, (2) What tasks can degrade, (3) How you'll measure both. If you can't answer these questions, you're not ready to fine-tune. A fine-tuned model without measured trade-offs is a model you don't understand.

For most task-specific fine-tuning, 1,000–5,000 high-quality examples is the sweet spot. Below 500, you risk underfitting. Above 10K, you get diminishing returns unless data quality remains excellent. A team that spends 80% of effort on data curation and 20% on training will outperform a team that does the opposite.

After training, show the model inputs that are similar but not identical to training examples. If performance is significantly worse than on training-like inputs, you're overfitting. A well-generalized model should handle reasonable variations without degradation.

Step 1: Normalize text (lowercase, strip whitespace, remove special chars) → Step 2: Exact dedup by hash → Step 3: Near-dedup by MinHash (threshold 0.7) → Step 4: Check for train/test/val leakage → Step 5: Log dedup stats (how many removed, from which sources). A 10% dedup rate is normal; >30% suggests data collection problems.

A fine-tuned model is a commodity — anyone can train one. A data pipeline that continuously improves from production usage is a moat. Teams that build the flywheel pull ahead over time; teams that treat fine-tuning as a one-time event get left behind.

The key insight: fine-tuning updates live in a low-rank subspace. You don't need to modify 16M parameters — the behavioral changes can be captured by two small matrices totaling 100K–500K parameters. The base model provides general intelligence; LoRA provides task-specific steering. At inference, A×B is merged into W — zero inference overhead.

Start with LoRA or QLoRA — they're the most tested, best supported, and work for 90%+ of use cases. Try DoRA if you need an extra 1–2% quality improvement and are willing to experiment. Everything else is research-stage or niche — don't use unless you have a specific reason and can validate the improvement on your eval set.

Production systems often route between base and fine-tuned models: use the fine-tuned model for in-domain queries where it excels, fall back to the base model for out-of-domain or uncertain cases. This preserves flexibility while gaining specialization. Implement confidence-based routing or query classification to decide which model handles each request.

Try LoRA first. Measure quality. If quality is not sufficient and you have evidence that more capacity is needed (ablation with higher rank doesn't help, or quality improves with more data but plateaus), then consider full fine-tuning. Full fine-tuning is a last resort, not a default.

Save checkpoints every 500–1000 steps. Keep the last 3 + best validation loss + best task metric. If training degrades, you can revert to an earlier checkpoint. For full fine-tuning, checkpoints are large (model size × 2 for optimizer states) — budget storage accordingly. Use save_only_model=True to save just weights if storage is tight.

Data format: {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}. Volume: 1K–5K high-quality examples for task-specific; 10K–100K for general instruction tuning. Quality bar: Every example should be one you'd be happy to ship in production. 1K excellent examples beats 10K mediocre ones.

The typical pipeline: SFT first → then DPO. SFT teaches the model what kind of responses to produce; DPO refines which responses are better. Add DPO when: (1) you have clear preference signals (human feedback, A/B test results), (2) SFT quality plateaus but you can still rank responses, (3) you need to reduce specific failure modes (safety, tone, verbosity) that are easier to express as "not this" than "do this."

90% of production fine-tuning is SFT-only. It's simple, it works, and it's what you should start with. DPO adds 5–15% quality improvement when you have good preference data and SFT has plateaued. RLHF is for frontier labs. Don't over-engineer your training objective — get SFT right first, add DPO if needed, and skip RLHF unless you have a very specific reason and resources.

Allocate at least 10% of your data curation effort to building and maintaining your golden test set. This is non-negotiable. A team that spends all effort on training data but has a weak eval set will ship bad models and not know it. The golden set is how you know if your fine-tuning is working.

Before trusting LLM-as-judge, validate against 50–100 human-labeled examples. Compute agreement rate (should be >80% for binary better/worse). If agreement is low, refine your evaluation prompt or criteria. Also test for position bias by running each comparison twice with swapped order — disagreement rate should be <10%.

Before instruction tuning (pre-InstructGPT era), LLMs required careful prompt engineering to produce useful outputs. Instruction tuning (SFT on instruction-response pairs) made models that try to help by default. This is the foundation of ChatGPT, Claude, and every modern assistant. If you're fine-tuning a base model, instruction tuning is almost always the first step.

1,000 high-quality, diverse examples often beats 50,000 similar examples. Why? Large models already have the capability — instruction tuning is about eliciting existing capability, not teaching new knowledge. Diversity shows the model the range of behaviors expected; quality shows it the standard to meet. Volume alone just overfits to a narrow distribution.

Don't use any single dataset as-is. Combine: (1) High-quality curated examples (200–500, manual or filtered) + (2) Task-diverse public dataset (5K–20K, filtered) + (3) Your domain-specific examples (as many as you have). Filter for quality, deduplicate, and shuffle. The blend matters more than any single source.

Modern base models already have substantial domain knowledge from their training corpus. Try domain SFT first — it's faster and cheaper. Add continued pre-training only if: (1) the model misuses domain terminology, (2) domain text is highly specialized and underrepresented in base model, (3) you have access to 100M+ tokens of domain text. For most use cases, SFT alone is sufficient.

Quality over quantity: 500–5,000 expert-curated examples beats 50K noisy ones. Task diversity: Cover all task types you'll use in production. Realistic inputs: Use real examples from your domain, not synthetic simplifications. Include edge cases: Hard examples, errors, ambiguous cases. Expert review: Every response should be verified by a domain expert.

Standard benchmarks give you a baseline, but custom evaluation on your actual production tasks is essential. Create a golden test set of 200–500 examples representing real use cases in your domain. Include expert review for high-stakes domains (medical, legal, finance). Compare to both base model AND to human expert or GPT-4 baseline to understand where your fine-tuned model wins and loses.

Privacy/compliance required? → Self-host or on-prem. Variable/bursty traffic? → Serverless. High constant throughput? → Dedicated vLLM cluster. Just need it to work? → Cloud API with fine-tuning (if available). Development/testing? → Ollama locally.

Production GPU (vLLM): AWQ 4-bit — best quality/speed tradeoff. Local/Edge (Ollama): GGUF Q4_K_M or Q5_K_M. Quality-critical: FP16 or INT8 — don't quantize. Memory-constrained: Q3_K_M or Q2 — significant quality loss, last resort. Rule of thumb: Test your specific task at each quantization level; losses vary by task.

If you don't have a formal registry, at minimum: (1) Store models in versioned cloud storage (S3, GCS) with naming convention (e.g., medical-llama-v2.3-2024-04-15/), (2) Keep a YAML/JSON manifest with model version → storage path → training run ID → eval metrics, (3) Document which version is in production. This beats scattered files and forgotten experiments.

Metrics: Prometheus + Grafana or Datadog. Logging: Structured logs → aggregator (Loki, CloudWatch, Datadog). Tracing: OpenTelemetry → Jaeger or vendor. LLM-specific: LangSmith, Langfuse, or custom sampled eval. Alerting: PagerDuty, Slack, email for critical metrics.

Month 1: You ship a fine-tuned model. Month 2: You've collected 1,000 examples of user corrections; you retrain and quality improves 5%. Month 3: The better model gets more usage, generating more feedback. Month 6: You have 10,000 curated examples and 20% better quality than month 1. Teams that build the flywheel pull ahead of teams that treat fine-tuning as a one-time event.