ROBATZ editorial · · 5 min read
Define the user outcome first
Three products can look like the same chat box while solving different problems. One finds the right policy document. Another explains how several policies apply to a question. A third converts a customer message into a consistent classification. Search, retrieval-augmented generation and fine-tuning are tools for different parts of those jobs; the interface alone does not tell you which to buy.
Write ten real requests and the ideal result for each. Should users receive documents, a composed answer, structured data or an action? Note whether they need source evidence, how often the information changes and what happens if the result is wrong. This small exercise makes an architecture discussion much more specific.
Use search when finding the source is the job
If users want to locate an invoice, a product specification or an exact policy, a good search experience may be sufficient. Filters, sensible result titles, previews and document access can matter more than adding a generated answer. Search can combine exact terms with meaning-based retrieval when that improves the task.
A hypothetical procurement team might need the current contract and its owner, rather than a paragraph summarizing it. Test whether a results list helps that person finish the job. Add generation when composing an answer genuinely saves effort and the added latency, review and error handling are justified. You can preserve direct access to the original source in either design.
Use RAG to bring relevant information into an answer
Retrieval-augmented generation, or RAG, finds relevant content and supplies it to a language model when it produces an answer. Microsoft describes this pattern as useful for private or changing information. An index can support keyword, vector or hybrid retrieval; RAG is a system pattern, not a requirement to choose one particular database.
For example, an internal assistant could retrieve approved product documentation and answer with links to supporting passages. The project still needs a process for refreshing documents, preserving access rules and recognizing insufficient evidence. RAG does not guarantee correct answers. If the retrieval step misses a crucial exception, a fluent explanation can still mislead the reader.
Consider fine-tuning for a measured behavior gap
Fine-tuning changes a model through additional training. It may be useful when you need more consistent behavior, output style or performance on a defined task. Microsoft's guidance distinguishes these goals from supplying fresh knowledge. A frequently updated handbook is usually better treated as a maintained information source than as something that must be retrained into a model each time it changes.
Before commissioning training, establish what prompting and examples can achieve. Ask whether you have enough representative, reviewed training data and a separate evaluation set. Clarify the cost of maintaining that dataset, repeating training and checking regressions. The business case should identify the specific behavior that is currently failing and the improvement needed to justify the work.
Compare complete experiences with the same tasks
Run candidate approaches against the same requests. Measure whether the correct source was retrieved, whether the answer is supported, whether access restrictions hold and whether users can complete the task. Also record response time, operating cost and the amount of correction required. A technically sophisticated architecture can still be the wrong product if users cannot inspect or trust its output.
Keep failure categories separate. Missing documents call for a different fix from poor ranking, incomplete permissions or a badly structured answer. Microsoft's RAG design guidance recommends evaluating individual stages as well as the final experience. This helps prevent buying a larger model to compensate for a content or retrieval problem it cannot reliably solve.
Make the first commitment small and testable
A useful first engagement produces a data inventory, a set of representative tasks, a baseline and an evidence-backed recommendation. It might conclude that better search is enough, that RAG is appropriate or that a combined approach deserves testing. Ask for the tradeoffs and maintenance responsibilities in writing, including how deleted documents and changed permissions reach the running system.
Bring the people who maintain the information into the decision. Someone must own document quality, access and updates after launch. The strongest architecture is the one your organization can operate as well as the one your engineers can build. Choose by demonstrated task performance, then expand the scope when the evidence supports it.
Apply this to your product
Explore our AI & machine learning development service for scope, deliverables and the decisions to make before starting.