Fine-tuning vs RAG vs Prompting

Welcome back to the NEXUS AI Literacy Series. This episode closes out our Memory and Knowledge track, and it's one of the most genuinely useful ones for anyone making real decisions about AI. Because here's a question that comes up constantly the moment a business gets serious: "We want the AI to do this specific thing for us — how do we actually make that happen?" And there are three different answers, three different levers you can pull. They're called prompting, RAG, and fine-tuning. People mix them up all the time, throw money at the wrong one, and get frustrated. By the end of this episode, you'll understand all three, and — more importantly — you'll know which to reach for and in what order. This is the kind of clarity that saves real time and real money.

Let me give you the one-sentence version of each up front, then we'll go deep. Prompting is changing what you say to the model. RAG is giving the model documents to look at. Fine-tuning is actually retraining the model itself. They go from cheapest and fastest to most expensive and involved — and the golden rule, which I'll keep coming back to, is: always try them in that order. Start with prompting. Most of the time, you never need to go further.

Let's start with prompting, because it's the most underrated. Prompting is simply the art of giving the model better instructions. Same model, no new technology — you just get more skilled at asking. And the leverage here is bigger than almost anyone expects. The difference between "write me a marketing email" and "you are a senior copywriter for a premium wrestling brand; write a punchy 80-word email to lapsed members, warm but direct, no corporate buzzwords, ending with a single clear call to action" — that's night and day, and it costs nothing but a little thought.

There's a particularly powerful prompting trick worth naming: giving examples. If you show the model two or three examples of exactly the kind of output you want before asking for a new one, its quality jumps dramatically. It pattern-matches off your examples. The industry calls this "few-shot prompting" — fancy term, simple idea: show, don't just tell. The analogy here is the simplest of all: prompting is just learning to ask a brilliant employee for what you want, clearly and with examples. You'd be amazed how often the "AI isn't good enough" problem is actually an "I asked vaguely" problem. So rule one: before you spend a dime on anything fancier, get the prompt right.

Now, prompting has a ceiling. No matter how well you ask, the model still only knows what it was trained on, and it still can't see your private documents. That's where RAG comes in — and we just spent a whole episode on it, so I'll be quick. RAG, remember, is the open-book exam: you connect the model to your documents so it can look things up and answer from real source material. You reach for RAG when the problem is about knowledge — when the model needs to know things it doesn't, like your company's policies, your product catalog, your customer records, or current information past its cutoff. The key insight for this episode is what RAG changes: it changes what the model knows, by giving it the right pages to read. It does not change how the model behaves or its fundamental skills — it just hands it the right reference material. Need it to know your stuff? That's RAG.

Then there's the heavy machinery: fine-tuning. Fine-tuning means taking an existing trained model and training it further on a batch of your own examples — actually nudging those internal dials we talked about, so the model itself changes. This is more involved, more expensive, and requires assembling a quality dataset of example inputs and outputs. So when is it actually worth it? Here's the crucial distinction that most people get wrong. You don't fine-tune to teach the model new facts — that's RAG's job, and fine-tuning is a bad, expensive way to do it. You fine-tune to teach the model a new behavior, skill, style, or format. When you need it to consistently respond in a very specific voice, or always output a rigid structured format, or handle a specialized task in a particular way that's hard to fully capture in a prompt — that's fine-tuning territory. RAG changes what it knows; fine-tuning changes how it behaves.

Let me give you the analogy that ties all three together, because it makes the whole decision obvious. Imagine you've just hired a brilliant new employee — sharp, well-educated, but brand new to your company. How do you get them productive?

First, you give them clear instructions for the task at hand. "Here's what I need, here's the tone, here's an example of a good one." That's prompting. Instant, free, and shocking how far it gets you with a smart hire.

Next, when they need to know company-specific things, you don't surgically alter their brain — you give them access to the company wiki and the file cabinet so they can look things up. That's RAG. They keep their general intelligence and now they can answer about your specific world.

Finally — only for a deep, specialized role they'll do over and over — you send them through weeks of intensive company training until a particular way of working becomes second nature, automatic, part of who they are on the job. That's fine-tuning. Expensive, slow, and absolutely worth it for the right specialized, high-volume role — and total overkill for a new hire you just need to write a few good emails.

So here's your decision framework, the actual takeaway you can use tomorrow. Always start with prompting — it's free, instant, and solves more than you'd think. If the issue is that the model doesn't know your specific or current information, add RAG. Only if you need a deeply consistent specialized behavior, style, or format that prompting can't reliably get you — and you have the volume to justify it — do you reach for fine-tuning. And here's the part nobody tells you: these aren't either-or. The most powerful real systems often use all three together — a fine-tuned model, that uses RAG to pull in fresh knowledge, driven by a well-crafted prompt. They're layers, not rivals.

One honest note to keep you sharp: people constantly reach for fine-tuning first because it sounds the most impressive and "real" — "we're training our own AI." It's usually the wrong instinct. Nine times out of ten, a better prompt plus good RAG gets you there faster, cheaper, and with far less to maintain. Fine-tuning is a powerful tool that's frequently the wrong one. Knowing that will save you from an expensive detour that a lot of companies take.

So let's bring it home. There are three ways to make a model do what you want, in order of cost and effort. Prompting changes what you say to it — start here, always; it's free and underrated, especially with examples. RAG changes what it knows — reach for it when the model needs your private or current information; it's the open book. Fine-tuning changes how it behaves — reach for it only for a deeply consistent specialized skill, style, or format, at enough volume to be worth it. New instructions, then access to the files, then intensive training — exactly how you'd onboard a brilliant new hire. Try them in that order, combine them when it helps, and don't pay for heavy machinery when a better question would do.

And that closes our Memory and Knowledge track. You now understand the model's desk, how it searches a library by meaning, how RAG grounds it in your real documents, why it hallucinates, and the three levers for shaping its behavior. That's a seriously complete picture of how AI actually gets put to work.

In the next track — the Agent Era — we make a leap. So far we've talked about AI that thinks and talks. Next, we give it hands. We start with tool use: how an AI goes from just answering questions to actually doing things in the world — searching the web, running calculations, sending emails, taking action. That shift is the biggest thing happening in AI right now. See you there.