Retrieval-Augmented Generation

Answer from the documentation you actually have

A documentation assistant is asked, “Can version 3 export CSV?” Its model may have learned general software patterns, but that does not establish what your current product supports. Retrieval-augmented generation, or RAG, first finds relevant external material and then includes it as evidence in a generation request. The answer should reflect the supplied documentation and expose the supporting sources.

An ordinary RAG request does not retrain the language model. You can update the document collection without changing its parameters, although the retrieval index and caches must also be updated. Retrieval can use keywords, embeddings or a combination; a vector database is one implementation choice, not the definition of RAG. If returning a matching page directly solves the task, a search interface may be sufficient without generation.

Prepare sources before answering questions

Ingestion turns approved source documents into retrievable records. Extract text, preserve headings and split it into coherent passages. Store a stable passage ID, document ID, source location, revision, version and access metadata with each record. A paragraph about export is misleading if its “enterprise accounts only” qualification is separated from it or its version heading is discarded.

Choose chunk boundaries using the document structure and the questions users ask, rather than an arbitrary character count alone. Small chunks can lose conditions; large chunks can waste context and combine unrelated claims. Overlap may preserve continuity but creates duplicate candidates. Inspect a few retrieved chunks manually and evaluate changes. Reject unsupported file formats and unapproved sources, and keep ingestion failures visible instead of indexing partial or corrupted text silently.

Retrieve only evidence the caller may use

For each query, establish the caller’s identity and allowed document scope before supplying passages to the model. Enforce access in the retrieval service or database policy, not in a prompt asking the model to hide other users’ material. Keep permission changes and deletions synchronized with indexed chunks and caches. Cache keys must reflect the relevant access scope, source revision and query configuration.

Within that authorized scope, retrieve candidates and check version and relevance. A reranker can reorder candidates using a different relevance model, but it cannot recover a missing source that was never retrieved. Keep the evidence budget bounded and include surrounding qualifications. If nothing supports the question, return an insufficient-evidence result or ask for a version rather than treating the nearest available passage as an answer.

Inspect a tiny retrieval pipeline

This local simulation has two fictional passages: public CSV documentation and a private nightly-export note. It uses exact word matching so the evidence flow is easy to inspect. This is not semantic search and does not generate an answer. The access filter runs before scoring, and citation IDs are resolved only against the authorized passages actually selected for this request.

Run the complete snippet in Node.js or a browser console. The checks show that user-a retrieves the public CSV passage, cannot retrieve the private nightly note, and cannot cite an invented ID. The returned source URL comes from the application’s source record. A real system needs authenticated identity, robust retrieval, content validation and current permissions instead of this small in-memory fixture.

const passages = [
  { id: 'doc-csv-v3', readers: ['public'], text: 'Version 3 supports CSV export.', url: 'https://example.com/docs/v3/export' },
  { id: 'note-nightly', readers: ['user-b'], text: 'Nightly export is available to a private beta cohort.', url: 'https://example.com/internal/nightly' },
];
function retrieve(userId, query) {
  const terms = new Set(query.toLowerCase().match(/[a-z0-9]+/g) || []);
  return passages
    .filter(p => p.readers.includes('public') || p.readers.includes(userId))
    .map(p => ({ ...p, score: (p.text.toLowerCase().match(/[a-z0-9]+/g) || [])
      .filter(word => terms.has(word)).length }))
    .filter(p => p.score > 0)
    .sort((a, b) => b.score - a.score || a.id.localeCompare(b.id))
    .slice(0, 2);
}
function citationUrl(id, evidence) {
  const passage = evidence.find(p => p.id === id);
  if (!passage) throw new Error('Unknown evidence ID');
  return passage.url;
}
const evidence = retrieve('user-a', 'CSV');
if (evidence.length !== 1 || evidence[0].id !== 'doc-csv-v3') throw new Error('Wrong evidence');
if (retrieve('user-a', 'nightly').length !== 0) throw new Error('Private evidence leaked');
if (citationUrl('doc-csv-v3', evidence) !== 'https://example.com/docs/v3/export') throw new Error('Wrong citation');
let rejected = false;
try { citationUrl('invented-id', evidence); } catch { rejected = true; }
if (!rejected) throw new Error('Invented citation accepted');
console.log('4 retrieval-boundary checks passed');

Give the model evidence, then check its use

Assemble the user’s question with selected passages labeled by their application-assigned IDs, titles and versions. Instruct the model to answer from those passages, identify missing evidence and associate factual claims with source IDs. Keep retrieved material clearly marked as data. A document may contain instructions such as “ignore the user”; labels help organize content, but application permissions remain the actual boundary against harmful actions.

Validate returned citation IDs against the evidence set and build links from stored metadata rather than accepting model-invented URLs. Then assess support: “Version 3 supports scheduled CSV exports” is not established by the public passage merely because the citation ID is real. Check each relevant claim against the passage, including conditions and negations. Expose uncertainty or conflicting sources rather than using citations as decoration.

Measure the stages separately

Before tuning, create questions with acceptable source passages and expected answer properties. Include synonyms, exact identifiers, conflicting versions and questions the collection cannot answer. Check whether supporting passages reach the candidate set; if they do not, inspect ingestion, filtering and retrieval. If they do, inspect context selection and generation. This avoids trying to fix every failure by changing the final prompt.

Evaluate source-supported claim rate, correct citations, appropriate abstentions and end-to-end usefulness. Keep held-out questions for final comparison so repeated tuning does not merely memorize the development set. Also test unauthorized queries with multiple accounts. A good answer score cannot compensate for disclosing a private passage, and an access-safe retrieval service can still return irrelevant public material.

Keep the evidence current and inspectable

Treat source updates as a data pipeline. Re-extract changed documents, replace or remove obsolete chunks, regenerate affected embeddings and invalidate relevant caches. Record document and embedding-model revisions so an unexpected answer can be traced to the evidence actually used. If an embedding model changes, rebuild compatible document vectors rather than comparing old vectors with a new query space.

Protect stored passages, vectors and logs according to their source sensitivity; embeddings are not anonymization. Retain enough provenance to investigate mistakes without collecting unnecessary private queries. Recheck retrieval and answer quality after material changes to documents, chunking or models. RAG makes external evidence available and inspectable, but it does not guarantee that sources are correct, that every useful source was found or that generation follows the evidence.

Sources & further reading

Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.

• Lewis et al.: Retrieval-Augmented Generation
• Supabase: RAG with permissions
• OWASP: RAG security