Three tools, three jobs, endless confusion. Here's the decision tree we actually use.
Start with prompting. Always. If the task is well-bounded and the base model is strong (GPT-4.1, Claude, Gemini 2.5), a good system prompt with few-shot examples solves 70% of problems.
When prompting breaks: facts move fast (news, prices, internal docs) or the model needs to look at your data. That's RAG. Retrieve the relevant chunks, stuff them in context, answer with citations. Most 'AI needs training on our data' requests are actually RAG requests.
When RAG breaks: the model doesn't know how to respond in your format, or the task needs consistent behavior across thousands of calls. That's fine-tuning. Teach the model your style, your format, your judgment — not facts.
Retraining (from scratch): almost never. Reserved for companies with proprietary corpora nobody else has and budgets to match.
Cost comparison: prompting ~$0. RAG ~$50/month (embeddings + vector DB). Fine-tuning $100-$10,000 per run depending on model size and data.
Latency: prompting and fine-tuning are fast. RAG adds retrieval latency (~50-200ms).
Our default: prompt → RAG → fine-tune, in that order. Most teams stop at step two and ship something great.