Separate knowledge from behaviour
A company assistant may need current policy information, a particular answer format and a way to respect access permissions. Those are different requirements. Before discussing model techniques, identify whether the problem is finding evidence, following a task pattern, producing a format or integrating with the right system. One technique does not automatically solve all of them.
What retrieval adds
Retrieval-augmented generation, or RAG, finds relevant material at request time and supplies it as context for an answer. This can help an application use approved documents and present source references. It also creates engineering work: ingestion, indexing, permissions, source updates, ranking and evaluation. A model cannot compensate for a retrieval step that misses the important evidence.
What fine-tuning changes
Fine-tuning adjusts a model using training examples. It can be considered for a well-defined behaviour or task when suitable examples and a reason to adapt the model exist. It is not a substitute for a document-permission system or a reliable process for keeping changing company facts current. Training also requires evaluation and a plan for maintaining the resulting model.
A practical decision guide
If users need answers from frequently changing internal documents, investigate retrieval and source management first. If the issue is consistent task behaviour or output format, test prompts and validation before deciding whether adaptation is needed. If the answer already exists in a structured system, a direct query or integration may be sufficient. Some applications combine techniques, but each addition should solve an observed problem.
Evaluate retrieval and answers separately
For a set of representative questions, record which sources should be found. Check whether the retrieval step returns them, then assess whether the answer correctly uses the available evidence. Test ambiguous questions and cases with no supporting source. A citation is helpful only when it actually supports the associated claim, and the interface should let users inspect it.
Keep permissions and freshness explicit
Decide how user identity affects retrieval, how updates and deletions propagate, and who owns each source. Test restricted documents with different roles. Agree what happens if a source is unavailable or inconsistent. These operational questions affect usefulness regardless of which model produces the final wording.
Begin with the smallest useful experiment
Choose a bounded collection of documents and real user questions. Establish a baseline with the existing search process. Compare retrieval approaches and answer behaviour against the same cases, including review time and running cost. Expand only when the experiment explains where the approach works and what still needs improvement.
Start with the information need
A discussion aid to adapt to your workflow.
| What needs to change? | What to evaluate |
|---|---|
| Access to changing company documents | Retrieval, source quality, permissions and freshness. |
| Consistent response style or structure | Instructions and examples first; then whether adaptation adds value. |
| Both current facts and specialised behaviour | Evaluate retrieval and behavioural adaptation separately. |
| Finding a record using an exact identifier | A direct query or conventional search may be sufficient. |
Make it specific
How does this apply to your business?
Bring us the workflow, your current systems and the decision you need to make.
Discuss your project