Retrieval or training: the question that decides your budget

---

Two completely different things, one sentence

"We want an AI trained on our documents" is the most common opening line I hear, and it describes two entirely different pieces of engineering that happen to sound identical when spoken aloud.

One of them is retrieval: the system searches your files, finds the passages that bear on the question, and hands them to the model to answer from. The model has learned nothing. It is reading, the way you would read a file before answering a client's email.

The other is training, which in practice means a LoRA adapter: you show a model several hundred examples of your finished work, and it adjusts until it produces documents that sound like yours. It has learned nothing factual about your firm. It has learned a manner.

These are not two grades of the same product. They solve different problems, they cost different amounts, and choosing the wrong one produces a system that fails in a way nobody can quite articulate. It works, it is just somehow not right.

The rule

Retrieval for what it needs to know. Training for how it should behave.

That is the whole thing, and it survives almost every case I have encountered.

If the sentence you would use to describe the problem contains the words find, look up, according to, which clause, what did we say last time, then you want retrieval.

If it contains sounds like us, our format, the way we always write it, the standard wording, then you want training.

A test you can apply before you call anyone

Ask this: if a new graduate joined tomorrow, would you fix this problem by giving them access to the file server, or by giving them a style guide and six months of supervision?

Access to the files is retrieval. Supervision is training.

It is a crude test and it is right most of the time, because it isolates the same distinction: knowledge you can look up versus judgement you have to absorb.

Why picking wrong is expensive in both directions

Training a model to answer factual questions about your firm is the more common mistake, and the more damaging one. A fine-tuned model does not store facts reliably. It stores tendencies. Ask it which standard applied to a job in 2023 and it will produce something with the right shape, the right tone, and the wrong number, delivered with complete confidence. It has learned what your answers look like, not what they are. It will also be wrong about anything that happened after training, and it cannot cite a source, because there wasn't one.

You will have paid four to six thousand dollars for a system that is worse at the job than a search box.

The reverse error is quieter. Firms bolt retrieval onto a general model and are mildly disappointed for a year. The facts are right. The documents are correct and completely characterless. They read like a competent stranger wrote them, because one did. Every draft needs a rewrite for tone, which eats most of the time it was supposed to save. That is the case where an adapter genuinely earns its price, and where I would happily take the work.

What each one actually costs you

Retrieval is mostly plumbing: getting your documents into a searchable form, keeping that current, and being careful about which of them a given person is allowed to see. The last part is where the real work is, and it is the part that gets skipped. It is also the piece I have written about separately, because permissions in the retrieval layer are one of the components in what actually runs on the box. A retrieval system that cheerfully quotes the partners' remuneration file to a summer clerk has not saved anyone time.

Training is mostly the dataset. Building it is thirty to fifty percent of the effort, and it is unglamorous: finding several hundred documents you are actually proud of, pairing each with the input it came from, and stripping the client details. The training run itself is the short part. Anyone selling you fine-tuning who has not asked hard questions about your document archive is selling you the short part.

The case for both, which is real but narrower than it sounds

Plenty of mature systems use both, and they compose cleanly: retrieval supplies the facts for this particular job, the adapter shapes how those facts are written up. A producer statement, a client letter, a file note: same structure every time, different specifics every time. That is exactly the shape where the two together beat either alone.

But that is the second or third build, not the first. Start with retrieval, live with it for a few months, and you will know precisely how much the tone problem is costing you, which is a much better basis for spending four thousand dollars than a guess made before anything was running. The same instinct applies to the hardware: falling prices, rising bills is the argument for not buying capacity before you have measured the need for it.

The uncomfortable disclosure

Fine-tuning is the more profitable service on my list, the more technically interesting one to build, and the one that is harder for a competitor to copy. Retrieval is closer to plumbing.

Which is exactly why you should be suspicious when anyone in this industry, including me, recommends it enthusiastically without having asked what you actually want the system to do. The incentive runs one way. The honest answer usually runs the other.