The need

Your teams spend time searching: in Confluence, in PDFs, in e-mails, in a colleague's head. A RAG assistant (Retrieval-Augmented Generation) answers their questions from your documents, and cites its sources so anyone can verify.

It is the most requested GenAI use case in companies. It is also the one where the gap between demo and production is widest: a RAG demo takes two days, a reliable RAG in production takes a real architecture.

The architecture

The system splits into two pipelines that live at different rhythms.

The ingestion pipeline turns your documents into a queryable base:

  • Extraction: every format (PDF, Word, Confluence pages, tickets) has its parser. Less glamorous than the LLM, and where half of all quality problems are born.
  • Chunking: documents are split into passages that respect their structure: sections, tables, lists. Splitting blindly through the middle of a table produces wrong answers downstream.
  • Hybrid indexing: every passage is indexed twice, as vectors (for meaning) and as keywords (for exact references: product names, codes, acronyms). Either one alone misses questions.

The answering pipeline handles each question:

  • Retrieval: both indexes are queried, results merged, then re-ranked by a model that keeps only genuinely relevant passages.
  • Generation: the LLM answers strictly from the passages provided, under a hard rule: cite every source, and say "I don't know" when the documents don't answer.
  • Citations: every claim links back to the source document. This is what turns a gadget into a work tool: the user can check.

The traps we know

  • The rotten corpus. A RAG answers from what you feed it. If three contradictory versions of a procedure coexist, it will cite one of the three at random. Cleaning the corpus is part of the project, not of the aftermath.
  • "Looks good to me." Ten hand-tested questions prove nothing. You need an evaluation set: real questions from your teams, with expected answers, replayed on every change.
  • Access rights. If a document is confidential, the assistant must neither cite it nor draw on it, depending on who asks. Permission filtering is designed in from day one, not after the first incident.
  • Silent drift. Documents change, the index ages, quality sinks without a sound. Without scheduled re-indexing and quality tracking over time, the system decays within months.

How we measure that it works

Three families of metrics, tracked continuously:

  • Retrieval quality: for each test question, do the right passages come up? (recall, re-ranking precision)
  • Answer quality: is the answer correct, sourced, and does it refuse to answer when it should? (evaluated automatically, human-checked by sample)
  • Cost and latency: cost per question, end-to-end response time, tracked per request in an observability tool.

What you keep at the end

The code in your repository, the index on your infrastructure, the evaluation set wired into your CI, and a team trained to keep the corpus alive. The system keeps running without us.