Retrieval-augmented generation succeeds when the assistant selects the right permitted source, extracts the correct passage, and knows when the evidence is too weak to answer. A vector search result alone does none of those things.
This guide is part of the NexisHub AI Infrastructure desk. It connects practical engineering, research, and product decisions to the wider NexisHub platform.
The operating idea
Imagine an internal knowledge base containing two pricing policies, one obsolete and one current. A semantic retriever may return both because their language is similar. Without effective dates and document authority, the model can confidently quote the wrong policy.
The design problem is not simply which embedding model performs best. It is how documents enter the index, how access permissions and versions travel with each passage, how search results are ranked and filtered, and how answers are checked against the retrieved sources.
NexisHub separates verified platform documentation, repeatable observation, and inference. No optimization can guarantee selection or citation by an external system.
Design the document record before splitting it
A useful source record includes a stable document ID, owner, version or effective date, access scope, canonical location, source type, and the most recent review status. Each searchable passage inherits those fields. This allows a query for a user's workspace to filter forbidden passages before they reach a ranking or generation stage.
Splitting by a fixed character count alone can detach a numerical value from its heading, qualification, or effective date. Prefer coherent sections with headings and controlled overlap where necessary. Preserve a reference back to the original document so people can inspect the whole statement.
A worked two-version retrieval test
Suppose a draft policy says a plan costs 20 units, while a later approved policy says 25 units. Both are synthetic examples. The retrieval layer should prefer the approved, currently effective policy when answering a current-price question, and it should explicitly cite the version consulted. If asked what changed, it may retrieve both, label their dates, and present the difference.
Write three expected outcomes for this fixture: a current pricing question cites only the effective source; a historical comparison distinguishes both sources; an unauthorised user's query returns neither restricted document. If the retriever cannot enforce those distinctions, improving prompt wording alone does not solve the data problem.
Use ranking as one stage, not the only judge
Keyword matching can recover exact identifiers; semantic matching can find paraphrases. A hybrid retrieval pipeline can combine their strengths and rerank with the question, passage and metadata. That choice should be evaluated on the organisation's own questions rather than selected from a benchmark claim alone.
Return a small evidence package with document identifiers, titles, excerpts, date and scope. Do not let the model silently create citations that were not supplied by retrieval. If passages conflict, surface the conflict or apply documented source precedence instead of guessing which fact is true.
Measure the whole answer chain
Create a fixture set containing exact-match queries, paraphrases, negations, absent facts, revoked access, stale documents, duplicated content and adversarial instructions inside retrieved text. Evaluate retrieval recall and relevance separately from grounded answer quality. A model may answer well despite retrieving poorly by relying on memorized general knowledge, which masks the problem.
A grounded-answer test asks whether each important statement is supported by a cited passage. Another check examines whether the system appropriately abstains when no permitted support exists. Track the failing stage so fixes target indexing, filters, ranking, context assembly or generation rather than rewriting everything.
Core principles
- Source authority comes firstStore access, version and provenance metadata with each passage.
- Retrieve only permitted evidenceFilter by tenant, ownership and publication status before showing context.
- Prefer documented source precedenceResolve stale conflicts with explicit version rules or surface uncertainty.
- Evaluate retrieval and answers separatelyA fluent answer is not proof of correct evidence selection.
A practical implementation workflow
Apply the work in a controlled sequence. Keep a baseline, name an owner, and define the evidence that will show whether each step was completed.
- 1. Build a versioned document registryDefine identifiers, owners, effective dates, source scope and correction paths.
- 2. Chunk meaningfullyPreserve headings, qualifiers, units and original document links.
- 3. Implement hybrid candidate selectionEvaluate lexical and semantic coverage for real question types.
- 4. Apply ranking with authority checksPrefer appropriate effective records and enforce permission filters.
- 5. Generate with bounded evidencePass attributable passages and require honest handling of missing support.
- 6. Run regression fixturesTest old versus new versions, unauthorised access, hallucinated citations and abstention.
Common mistakes
Indexing unpublished drafts with live policy
Similar text can obscure which version has authority.
Treating embedding similarity as truth
Similarity does not establish freshness, correctness or rights to access.
Evaluating only final prose
A persuasive answer can conceal wrong retrieval and unsupported claims.
How to measure it responsibly
Use separate retrieval metrics for known relevant documents, citation faithfulness checks for generated claims, and abstention tests when sources are missing. Document test-set selection and source snapshot dates.
Inspect security and privacy cases: the desired unauthorized retrieval count is zero, but a passing limited test does not prove tenant isolation is complete across all execution paths.
Keep observed outputs, diagnostic scores, inferred causes, and business outcomes in separate fields. A modelled score is not a citation, and correlation is not proof of cause.
What comes next
As model context windows grow, disciplined source versioning and authorization will still be needed. More context is not inherently more reliable context.
The durable response is to build pages that are accessible, semantically explicit, useful outside their original layout, and backed by evidence a reader can inspect.
Key takeaways
01Record document provenance, version and access before indexing.
02Preserve the context needed to interpret a passage.
03Test historical conflicts and absent evidence explicitly.
04A citation must lead to the source actually used.
Frequently asked questions
Should I use only vector search for RAG?
Not necessarily. Exact identifiers and domain terminology often benefit from lexical search or hybrid ranking.
Can the model choose the newest policy without metadata?
It may guess. Provide effective dates and an explicit authority rule instead.
What should happen when no permitted source answers a question?
Say that reliable support was not found, and offer an appropriate next step rather than inventing a citation.
References and further reading
Turn a useful idea into a maintainable product.
Nexis Studio helps organisations move from problem definition to product design, engineering, launch, and long-term support with clear technical and commercial boundaries.
Explore Nexis Studio