RAG Architectures — Swarnim Tiwari

01

How is each RAG architecture structured?

Retrieval-augmented generation solves the same problem four different ways: connect a language model to a knowledge base so it can answer questions grounded in that knowledge rather than in its training data alone. The architectural differences between the four approaches determine what kinds of questions each can answer, at what cost and where each one breaks.

View

The Baseline · Fast to Build · Limited Ceiling

Naive RAG

Index, Retrieve, Generate

Pipeline

Chunk documents (fixed size)

Embed each chunk

Store in vector database

Embed query, retrieve top-k

Generate with retrieved context

01Three-step pipeline: index (chunk documents and embed each chunk into a vector database), retrieve (embed the user's query and find the most semantically similar chunks), generate (pass the query plus retrieved chunks to the LLM and produce an answer). This is the simplest implementation of retrieval-augmented generation. It requires no special framework knowledge and can be built in under 100 lines of code with a vector store client and an LLM SDK.
02Chunking strategy is the most impactful single decision in naive RAG. Fixed size chunking splits documents into equal token blocks, typically 256 to 512 tokens with 10 to 20% overlap between consecutive chunks. The overlap prevents information loss at chunk boundaries by ensuring the same content appears at the end of one chunk and the beginning of the next. Chunk size trades retrieval recall against retrieval precision: smaller chunks retrieve more precisely but may miss context spanning multiple paragraphs; larger chunks preserve context but dilute precision.
03The embedding model determines the semantic search quality ceiling for the entire pipeline. For general domain text, the common production choices as of late 2026 include OpenAI text-embedding-3-large (3072 dimensions), Cohere embed-v3, BGE-M3 and E5-Mistral. Model performance varies significantly by domain: a model performing well on general web text may perform poorly on specialized medical or legal content. Selecting the embedding model on representative samples of the actual document collection and actual query distribution, not on generic benchmarks is the correct evaluation approach.
04The generation step receives three inputs: the user's original query, the top-k retrieved chunks (typically 3 to 5), and a prompt template instructing the model how to use the retrieved context. The model synthesizes an answer grounded in the retrieved material. Naive RAG provides no mechanism for the model to request additional retrieval, verify retrieved content quality or decide whether retrieval was necessary for the current query. Every query follows the same fixed pipeline regardless of its complexity or its relationship to the knowledge base.
05Failure modes appear clearly at scale: as the document collection grows, average retrieval precision decreases because the embedding space becomes denser and more similar documents compete for the top-k slots. Questions requiring synthesis across multiple documents or multiple distant sections of a long document cannot be answered by single-pass top-k retrieval because the necessary context is distributed across more than k chunks. Questions containing ambiguous terms retrieve high-similarity but low-relevance results because the retrieval step has no mechanism to resolve the ambiguity before searching.

Naive RAG is the correct starting point for any new RAG system, not because it is the best architecture but because it exposes the specific failure modes that should drive the next architectural investment. A team that goes directly to modular or agentic RAG without building naive RAG first cannot identify whether the investment in complexity is justified. The gap between naive RAG performance and required performance, measured on a representative evaluation set, should be the only input to the decision about which more complex architecture to adopt next.

Pre and Post-Retrieval Optimization · Better Precision

Advanced RAG

Wrapped Pipeline with Optimization Layers

Pipeline

Semantic chunking

Query rewriting / HyDE

Hybrid dense + sparse retrieval

Re-ranking (cross-encoder)

Context compression

01Advanced RAG adds optimization layers before and after the core retrieval step rather than replacing the core pipeline. Pre-retrieval enhancements improve what is indexed (better chunking, better document representation, richer metadata) and how queries are formed before they hit the index (query rewriting, query decomposition, query expansion). Post-retrieval enhancements improve what is injected into the generation context after retrieval runs (re-ranking, compression, result fusion). The naive pipeline remains the core; advanced RAG wraps it with measured improvements.
02Query rewriting transforms the user's literal query into one or more queries better suited for semantic retrieval. Hypothetical Document Embedding (HyDE) generates a hypothetical answer to the query and embeds that answer rather than the original question, reasoning that the answer will be closer in embedding space to documents containing real answers than the question is. Multi query generation produces several restatements of the same question to expand retrieval coverage across different phrasings. Step-back prompting extracts the underlying concept behind a specific question and uses the concept as an additional retrieval query to surface foundational documents.
03Re-ranking after retrieval: retrieve more candidates than needed — top-20 or top-50 — and then apply a cross-encoder re-ranker to score each candidate against the query with higher precision than bi-encoder embedding similarity. Cross-encoders process the query and each candidate document together in one forward pass, producing relevance scores that account for the interaction between query and document rather than their independent embeddings. The re-ranked top-k (typically top-3 or top-5) goes to the generator. Re-ranking consistently improves answer quality by 10 to 20% on standard benchmarks at the cost of 50 to 200ms additional latency.
04Context compression after retrieval: instead of injecting the full retrieved chunks into the generation context, extract only the sentences most relevant to the query from each retrieved chunk. This reduces total tokens injected while preserving key information, leaving more context window budget for the generator's reasoning. LLMLingua and similar compression tools apply this extraction step automatically. Compression adds latency and introduces a new failure mode: the compressor may remove sentences relevant to the final answer but not obviously related to the literal query text.
05Each optimization in advanced RAG addresses a specific measured failure mode of naive RAG. Adding all optimizations simultaneously without measuring each component's contribution produces a system that is more expensive and slower than naive RAG with unclear quality benefits. The correct approach is additive: add HyDE and measure its impact on a representative evaluation set, add re-ranking and measure again, add compression and measure again. Each addition should be justified by a measurable quality improvement that exceeds its latency and cost cost.

Advanced RAG is not an architecture, it is a collection of improvements that each address a specific failure mode of naive RAG. The right combination of improvements depends entirely on which failure modes matter for the specific application. A question-answering system with short, precise factual queries may benefit most from better chunking and re-ranking. A complex analytical query system may benefit most from multi-query expansion and result fusion. No single advanced RAG configuration is optimal across all applications. Measure before adding.

Composable Modules · Adaptive Routing · Multiple Backends

Modular RAG

Configurable Module Graph

Pipeline

Router (query classification)

Search module (multi-backend)

Fusion module (result merging)

Re-ranking module

Generation module

01Modular RAG reconfigures the fixed pipeline into composable modules with defined interfaces. Search, memory access, result fusion, re-ranking, compression, and generation are independent components that can be combined, reordered, or substituted based on query type and task requirements. Different queries route through different module combinations. Factual queries follow a simple retrieve-and-generate path. Analytical queries route through multi-query expansion and result fusion. Conversational queries pass through a memory module before hitting the retrieval index. The architecture trades simplicity for adaptability.
02Adaptive routing: a routing module classifies each incoming query and directs it to the appropriate module combination. Query classification uses either a small classifier model trained on labeled query types or an LLM prompt that categorizes the query before routing. The routing decision adds 20 to 50ms latency per query but reduces overall pipeline cost by directing simple queries through cheap module chains and complex queries through expensive ones. Without routing, every query pays the cost of the most expensive module combination regardless of whether that complexity is needed.
03Hybrid retrieval across multiple backends: modular RAG naturally accommodates multiple retrieval sources queried simultaneously. A single query might retrieve from a vector store for semantic search, a BM25 index for exact keyword matching, a structured SQL database for numerical data, and an agent memory store for personal context. The fusion module combines results from all backends into a ranked list using reciprocal rank fusion or another multi-source merging strategy. Supporting multiple retrieval backends is architecturally natural in modular RAG because the interface-based module design isolates backend-specific logic from the rest of the pipeline.
04Index composition alongside multi-source retrieval: modular RAG systems often maintain a relationship graph between documents — citations, version updates, topical co-occurrence — alongside the primary vector index. When seed documents are identified through semantic search, the relationship graph expands the initial retrieval set by surfacing related documents that were not directly similar to the initial query. This graph-augmented expansion addresses a specific class of multi-hop questions where the necessary documents are structurally adjacent but semantically distant from the query.
05The operational cost of modularity: each additional module adds latency, adds potential failure points, and adds configuration complexity that must be maintained. A system with routing, multi-query expansion, multi-backend retrieval, fusion, re-ranking, and compression has six distinct places where misconfiguration or quality degradation can reduce answer quality. Modular RAG systems require more sophisticated evaluation infrastructure than simpler pipelines — each module's quality contribution must be measurable independently to justify the added complexity. The investment is justified when the quality improvement over advanced RAG is measurable and the query diversity is high enough that different module combinations genuinely serve different query types better.

Modular RAG solves the problem of having one application that serves many different query types, each of which would benefit from a different pipeline configuration. If an application always receives the same type of query, the routing module adds cost without adding value — advanced RAG with a fixed pipeline optimized for that query type is more efficient. If an application serves highly diverse query types, modular RAG's routing-based optimization justifies its complexity. The decision between advanced RAG and modular RAG should be driven by measured query diversity, not by architectural preference.

Model-Controlled Retrieval · Iterative · Transparent

Agentic RAG

Agent Decides When and What to Retrieve

Pipeline

Agent receives query

Agent calls retrieval tools

Agent evaluates retrieved content

Agent retrieves again if needed

Agent generates grounded answer

01Agentic RAG gives the language model control over the retrieval process. Instead of running a fixed retrieval pipeline before generation, the agent decides whether to retrieve, what query to use, which knowledge source to target, and how many rounds of retrieval are necessary to answer the question confidently. Retrieval tools are part of the agent's action space rather than infrastructure that runs automatically before the agent sees the query. The agent is an active participant in information gathering rather than a passive consumer of pre-retrieved context.
02Iterative retrieval is the capability that separates agentic RAG from all fixed pipeline approaches. When the first retrieval round surfaces documents that raise new questions or reveal gaps in the retrieved context, the agent formulates new search queries based on what it learned and retrieves again. Multi-hop questions that require connecting information from multiple sources in a specific sequence — "what was the basis for the policy changed in the 2024 review that affected the teams involved in the 2023 incident" — require this kind of iterative retrieval reasoning. Fixed pipeline RAG cannot handle these questions reliably because it cannot plan the sequence of retrieval steps needed to assemble the necessary context.
03Retrieval transparency through tool-calling traces: the agent calls retrieval tools as function calls or tool_use blocks within its reasoning trace. Every retrieval decision — the query formulated, the source targeted, the results received — appears in the trace alongside the reasoning that preceded it. This transparency is not available in any fixed pipeline approach, where retrieval decisions are made by the infrastructure before the agent sees anything. When agentic RAG produces wrong answers, the reasoning trace shows exactly which retrieval queries were made, what they returned, and how the agent used those results — making debugging faster and evaluation more targeted.
04Self-RAG formalizes the self-evaluation component of agentic retrieval: reflection tokens in the model's output explicitly assess retrieval necessity (was it necessary to retrieve at all?), retrieved document relevance (is this document actually relevant to the question?), and answer support (does the retrieved context actually support the answer I am generating?). These explicit quality assessments before committing to a final answer reduce hallucination rates in domains where confidently wrong answers are more harmful than uncertain correct ones. Self-RAG was demonstrated to outperform standard RAG and instruction-tuned models on factual accuracy benchmarks.
05The cost of agent-controlled retrieval: multiple iterative retrieval rounds add latency that compounds with the number of rounds the agent takes. An agent running three rounds of retrieval and self-reflection before answering adds 3x the retrieval latency plus multiple additional LLM calls for the reasoning steps. Agentic RAG is cost-justified when answer quality on complex multi-hop questions genuinely requires iterative reasoning and when the latency increase is acceptable for the use case. For simple factual queries where one retrieval round is sufficient, advanced RAG with a fixed pipeline produces equivalent quality at a fraction of the latency and cost.

Agentic RAG is not an improvement on modular RAG — it is a different architectural philosophy about who decides how retrieval works. Fixed pipeline approaches (naive, advanced, modular) separate the retrieval decision from the model that uses the retrieved content. Agentic RAG collapses that separation. The benefit is retrieval decisions that are informed by the model's understanding of the question and its evolving context. The cost is latency, cost, and the dependence on the model's judgment quality for retrieval decisions. The right choice depends on whether the model's judgment about retrieval is better than a well-configured fixed pipeline for the specific query types the application handles.

02

How does each approach prepare and index documents?

No retrieval algorithm can recover information that was lost during indexing. The maximum retrieval quality is bounded by the quality of the indexed representation. Chunking strategy, embedding model, metadata design, and index structure together set the ceiling for every subsequent retrieval and generation step.

View

The Baseline

Naive RAG

Fixed-Size Chunks, Flat Index

Index Pattern

Fixed-size chunks (256-512 tokens)

Overlap (10-20%)

Single vector namespace

Minimal metadata

Batch indexing

01Fixed-size chunking with overlap is the standard naive RAG indexing approach. Split documents into chunks of N tokens (256, 512, or 1024 are common choices) with M percent overlap between consecutive chunks. The overlap prevents information loss at boundaries — a sentence split across two chunks appears in both, giving retrieval a second chance to surface that content. Every production RAG system should test multiple chunk sizes against a representative question set before committing to one, because the optimal size varies with document type and query distribution.
02Flat index structure: all chunks from all documents go into a single vector store namespace with no hierarchical organisation. This is fast to build and easy to maintain but degrades in search quality as the collection grows large or diverse. When the same index contains technical documentation, customer emails, and legal contracts simultaneously, the semantic distances between different content types create retrieval noise — a query about contract terms may surface documentation results because formal, precise language appears in both categories with similar embedding representations.
03Embedding storage per chunk: the embedding vector (1536 to 3072 dimensions for modern models), the original chunk text, and minimal metadata (document ID, chunk index, source URL). Naive RAG implementations frequently skip storing metadata that would enable filtering at retrieval time: date, author, document type, version, and topic category. The absence of metadata is typically invisible during development when the collection is small and homogeneous, and becomes a retrieval quality limitation at production scale when the collection is large and diverse.
04Batch indexing pipeline: process all documents through the chunk-embed-store pipeline in sequence at collection creation time, then re-run the full pipeline when the collection is updated. Naive RAG implementations often have no incremental update mechanism — adding one new document triggers re-indexing of the full collection to ensure consistency. At small scales this is acceptable. At large scales it produces indexing jobs that take hours, during which the index may be stale or unavailable, creating a production operational burden that was not visible during development.
05The indexing quality ceiling is the most important concept for teams starting with naive RAG: no retrieval improvement after indexing can recover information destroyed during indexing. Documents split at semantically meaningless token boundaries, embedded with a model that does not represent the domain vocabulary well, and stored without useful filtering metadata cannot be retrieved effectively regardless of how sophisticated the retrieval algorithm is. Investing in better chunking and embedding before tuning retrieval algorithms consistently produces larger quality improvements per engineering hour.

Naive RAG indexing treats all documents the same way regardless of their structure, content type, or typical query pattern. Technical documentation with precise terminology indexes the same way as conversational customer support transcripts. Legal contracts with defined terms index the same way as marketing copy with colloquial language. This uniformity is the source of both naive RAG's simplicity and its quality ceiling — the same strategy cannot be optimal for all document types simultaneously, and any application with diverse document types will hit this ceiling quickly.

Pre and Post-Retrieval Optimization

Advanced RAG

Semantic Chunks, Hierarchical Index, Rich Metadata

Index Pattern

Semantic chunking (boundary detection)

Hierarchical parent-child index

Multi-vector (dense + sparse)

LLM-extracted metadata

Incremental updates

01Semantic chunking replaces fixed token boundaries with natural topic boundaries detected by embedding similarity. Compute the embedding of each sentence in a document, measure cosine similarity between adjacent sentence embeddings, and place chunk boundaries where similarity drops sharply — indicating a topic transition. Semantic chunks represent complete thoughts or arguments rather than arbitrary token windows. Independent evaluations consistently show 15 to 25% improvement in retrieval precision from semantic versus fixed-size chunking, at the cost of higher indexing computation and variable chunk sizes that require more careful vector store configuration.
02Hierarchical indexing addresses the fundamental chunk size trade-off: build two indexes at different granularities simultaneously. A fine-grained index contains small, precise chunks (128 to 256 tokens) used for accurate retrieval. A coarse-grained index contains larger, context-preserving parent chunks (512 to 1024 tokens) or document-level summaries. When a fine-grained chunk is retrieved, the system automatically expands it to include its parent chunk before injection into the generation context. This parent-child retrieval pattern combines the precision advantage of small chunks with the context richness of larger units — addressing the core chunk size dilemma of naive RAG.
03Metadata enrichment using an LLM at index time: for each document or section, extract and store topic summary, entity list, date references, document type, and a content quality score before storing the chunk. This enriched metadata enables pre-filtering at retrieval time — a query about events in 2025 can filter to 2025-dated documents before semantic search runs, dramatically reducing the search space and improving precision. The metadata extraction adds one LLM call per document section at index time, which is acceptable for periodic batch indexing but must be budgeted for large collections.
04Multi-vector representation: store both a dense embedding (from a neural embedding model) and a sparse representation (from BM25 or SPLADE) for every chunk. Hybrid search using both representations at retrieval time consistently outperforms either representation alone by 10 to 20% in recall at k on standard RAG benchmarks. The dense vector captures semantic meaning and handles synonyms; the sparse vector captures exact terminology and handles domain-specific jargon not well-represented in the embedding model's training data. The cost is approximately doubled index storage and a more complex retrieval query, both of which are manageable for most production deployments.
05Incremental index updates rather than full re-indexing: new documents are processed and added without rebuilding the existing index. Soft deletion marks removed documents as inactive in metadata rather than physically deleting chunks from the index, preventing fragmentation. Updated document sections trigger targeted re-indexing of only the changed sections rather than the full document. These incremental update capabilities are less glamorous than chunking strategy or re-ranking algorithms, but maintaining index freshness without expensive full re-indexing is one of the most practically important production engineering achievements in large-scale advanced RAG deployments.

Advanced RAG indexing represents the observation that document preparation deserves as much engineering investment as retrieval algorithm design. A well-indexed collection with semantic chunks, hierarchical structure, rich metadata, and hybrid representations will outperform a poorly-indexed collection with sophisticated retrieval algorithms. Teams that reach the quality ceiling of naive RAG and immediately invest in better retrieval — re-ranking, multi-query, HyDE — without first addressing indexing quality are optimizing the wrong layer. Better indexing improves the input quality for all subsequent retrieval steps simultaneously.

Composable Modules

Modular RAG

Per-Collection Strategy, Multi-Index, Relationship Graph

Index Pattern

Per-collection chunking configuration

Multiple parallel indexes

Document relationship graph

Dynamic index selection

Continuous quality evaluation

01Per-collection indexing configuration: modular RAG applies different chunking strategies, embedding models, and metadata schemas to different document collections based on their content type and typical query pattern. Technical documentation is indexed with semantic chunking and dense embeddings optimized for precise technical terminology. Customer support transcripts are indexed with conversational chunking that preserves dialogue structure. Structured product catalogs are indexed into a searchable database alongside the vector index. One indexing strategy serving all collection types is replaced by optimal strategies per collection type.
02Multiple parallel indexes with different retrieval characteristics: alongside the primary dense vector index, modular RAG maintains a sparse BM25 index for keyword-exact retrieval, a graph database index for structured relationship queries, and optionally a table-structured index for numerical data. The routing module determines which indexes are queried for each incoming request. A question combining semantic context with exact terminology uses both the dense and sparse indexes. A question about organizational structure uses the graph index. Dynamic index selection by query type produces better per-query precision than searching all indexes for every query.
03Document relationship graph alongside the vector index: encode document citations, version relationships, topical co-occurrence, and explicit authorial links as edges in a graph indexed separately from the chunk embeddings. After semantic search identifies seed documents, graph traversal expands the retrieval set by following relationship edges to related documents that may not be semantically similar to the query but are structurally adjacent to the relevant documents. This two-phase retrieval — semantic seed then graph expansion — handles a specific class of multi-hop queries that requires documents connected by structure rather than by semantic similarity.
04Dynamic index selection by query classification: the routing module classifies each incoming query and directs it to the appropriate index combination before retrieval runs. A policy question routes to the policy document index. A recent event question routes to the time-filtered news index. A cross-domain question routes to multiple indexes simultaneously. Dynamic routing reduces noise in each individual retrieval operation by ensuring that only relevant collections are searched, improving both precision and latency compared to searching all available indexes for every query regardless of relevance.
05Continuous index quality evaluation: when new documents are indexed, automated evaluation checks whether retrieval quality for a representative question set improves, stays the same, or degrades. This continuous quality monitoring detects indexing regressions before they reach production users. A newly indexed document formatted unusually, or containing terminology that shifts embedding space representations, can degrade retrieval quality for other documents in the same collection. Without automated continuous evaluation, this degradation reaches users before the team detects it. With it, indexing changes can be gated on measured quality impact.

Modular RAG's indexing philosophy treats the knowledge base as a portfolio of collections with different characteristics rather than a single homogeneous store. This mirrors how knowledge is actually structured in most organisations: some information is structured and queryable, some is narrative and semantic, some is relational and graph-structured, some is time-sensitive and needs temporal filtering. Forcing all of this into a single flat vector index produces mediocre retrieval across all types. Per-collection optimized indexing produces strong retrieval for each type at the cost of managing multiple index systems.

Model-Controlled Retrieval

Agentic RAG

Tool-Described Indexes, Knowledge Source Registry

Index Pattern

Tool description as index documentation

Knowledge source registry

Diverse backend types

Agent-readable schemas

Bursty sequential load pattern

01Tool descriptions are the indexing documentation in agentic RAG. The agent needs to know what indexes exist and how to query them before it can make good retrieval decisions. The tool specification for each knowledge source — what it contains, what types of queries it is optimized for, what metadata filters are available, what the return format looks like — is as important as the quality of the index itself. A high-quality knowledge base with a vague tool description ("search documents") produces worse agent retrieval behavior than the same knowledge base with a precise, example-rich tool description that clearly communicates when and how to use each source.
02Knowledge source registry as dynamic tool discovery: as the number of available knowledge sources grows, agentic RAG systems benefit from a registry that maintains descriptions, usage examples, and quality metadata for each available source. The agent queries the registry to discover available tools before deciding which to call, rather than receiving a fixed tool list in the system prompt. Dynamic discovery allows new knowledge sources to be added without system prompt changes and prevents tool list bloat — a system prompt containing 30 tool descriptions is significantly harder for the agent to use correctly than one containing 5 well-described tools relevant to the current task.
03Diverse backend types are natural in agentic RAG because the agent calls each source through a uniform tool interface regardless of the backend technology. The agent calls search_policy_documents with a query string. The backend implementation handles whether that query goes to a vector store, a BM25 index, a SQL database, or an external API. Backend diversity is managed by tool implementation rather than by the agent's reasoning. Adding a new knowledge source means implementing a new tool with a clear description — the agent's retrieval behavior adapts automatically once the tool is available and well-described.
04Index freshness is more important for agentic RAG than for fixed pipeline RAG because the agent makes retrieval decisions based on its model of what each knowledge source contains. If a knowledge source is described as containing current inventory data but the underlying index is 48 hours stale, the agent may retrieve and use that data as if it were current, producing confidently wrong answers. The tool description should include freshness information (last updated timestamp, update frequency) so the agent can reason about whether the source is appropriate for time-sensitive queries.
05Vector database load patterns for agentic workloads differ from fixed pipeline patterns. Fixed pipeline RAG produces uniform concurrent retrieval — every session makes exactly one retrieval call per turn, distributed evenly across sessions. Agentic RAG produces bursty sequential retrieval — complex queries may trigger 5 to 10 sequential retrieval calls within one session while simple queries trigger only one. The same vector database configuration that handles uniform high-concurrency fixed pipeline traffic well may produce high tail latency under the sequential bursts of agentic workloads. Production load testing should use traffic patterns representative of actual agentic workloads rather than standard uniform-query benchmarks.

Agentic RAG places the knowledge source design problem in a new frame: instead of designing an optimal retrieval pipeline, design a set of well-described, composable knowledge sources that an intelligent agent can combine as needed for any given query. The quality ceiling shifts from retrieval algorithm quality to tool description quality and knowledge source coverage. A team that builds excellent knowledge sources with excellent tool descriptions empowers the agent to handle query types that were not anticipated at design time. A team that builds excellent pipelines for anticipated query types hits a ceiling when unanticipated query types arrive in production.

03

How does each approach retrieve relevant content?

Retrieval quality determines whether the generator has accurate, relevant source material or misleading, tangentially-related noise. The same generator with different retrieval quality produces dramatically different answer quality. Retrieval is where the indexed knowledge connects to the current question — and where most of the quality difference between RAG approaches is actually produced.

View

The Baseline

Naive RAG

Single-Pass Semantic Search

Retrieval Pattern

Embed query as-is

Cosine similarity search

Top-k chunks returned

No re-ranking

No query transformation

01Single-pass semantic search: embed the user's query using the same embedding model used during indexing, compute cosine similarity between the query embedding and all stored chunk embeddings, and return the top-k highest-similarity chunks. This single retrieval call completes in 10 to 50 milliseconds for typical vector store configurations. The result is the k chunks whose embedding representations are most similar to the query embedding — which correlates with but does not guarantee topical relevance to the actual question.
02The vocabulary mismatch problem: naive RAG retrieval finds chunks that are similar in embedding space to the query. When the user's question uses different terminology than the indexed documents — asking about "employee records" when the documents use "personnel files," or asking about a person by nickname when documents use their full name — embedding similarity may fail to surface the most relevant chunks. Exact keyword presence, which embedding similarity ignores, is sometimes the most reliable indicator of relevance for precise terminology queries.
03Context window budget for retrieved content: naive RAG passes all top-k chunks to the generator without filtering or compression. If k is 5 and each chunk is 512 tokens, the retrieved context occupies 2,560 tokens before the query itself is added to the context window. For models with large context windows, this budget is trivially available. For models with smaller windows or for systems where the context window budget must be shared with other content (system prompt, conversation history, memory), a fixed k with full chunk injection can exhaust the context budget on a single retrieval call.
04Retrieval precision degrades with collection size. In a collection of 1,000 documents, the top-5 retrieval result set contains a high proportion of genuinely relevant chunks because the embedding space is sparsely populated — most queries have few near-neighbor chunks. In a collection of 100,000 documents, many chunks occupy nearby regions of the embedding space, and the top-5 result set may contain chunks that are semantically similar to the query but do not actually answer it. Retrieval precision at scale is a function of collection density, not just retrieval algorithm quality.
05No query-time correction: naive RAG has no mechanism for the retrieval step to adapt if the first query returns poor results. There is no feedback loop between retrieval quality and query formulation. If the user asks an ambiguous question, the ambiguous query embedding retrieves ambiguous results, and the generator must synthesize an answer from potentially misleading context without any signal that the retrieval step underperformed. All retrieval quality improvements must be implemented at the algorithm level at build time, not at query time during serving.

Naive RAG retrieval's quality ceiling is visible in the gap between what the best possible embedding similarity can achieve and what a knowledgeable human expert would retrieve for the same query. The human expert would resolve ambiguous queries, use multiple search strategies for complex questions, and verify that retrieved results actually answer the question before proceeding. Naive RAG does none of these things. Every subsequent RAG architecture adds back one or more of these human retrieval behaviors in a mechanised form.

Pre and Post-Retrieval Optimization

Advanced RAG

Query Transformation + Re-ranking

Retrieval Pattern

HyDE / multi-query expansion

Hybrid dense + sparse search

Over-retrieve (top-20 to 50)

Cross-encoder re-ranking

Reciprocal rank fusion

01Query transformation before retrieval: HyDE generates a hypothetical answer to the user's question and uses the answer embedding for retrieval rather than the question embedding. The reasoning: a hypothetical answer is more similar in embedding space to actual answer-containing documents than the original question is. Multi-query expansion generates 3 to 5 restatements of the same question using different vocabulary and different framings, then retrieves for each restatement independently and merges results. Either technique produces higher recall than single-query literal embedding, particularly for short or ambiguous questions.
02Hybrid retrieval combines dense vector search with sparse keyword search in a single retrieval operation. Reciprocal rank fusion merges the two result lists by computing a combined score from each document's rank position in the dense results and its rank position in the sparse results. Dense search handles conceptual similarity; sparse search handles exact terminology. The hybrid combination consistently produces better recall than either individual approach across query types. The improvement is most pronounced for domain-specific terminology that is underrepresented in the embedding model's training data — a common situation for enterprise document collections.
03Over-retrieve and re-rank: retrieve more candidates than the final k needed (top-20, top-30, or top-50 depending on collection characteristics) and then apply a cross-encoder re-ranker that scores each candidate against the query with higher precision than bi-encoder embedding similarity. Cross-encoders process query and document together rather than independently, producing relevance scores that account for fine-grained semantic overlap. The re-ranker's top-k output goes to the generator. Over-retrieve-then-rerank consistently improves answer quality by 10 to 20% on standard RAG benchmarks at the cost of additional re-ranking latency, typically 50 to 200ms for a top-20 re-ranking operation.
04Parent chunk expansion after retrieval: after identifying precise child chunks through semantic search, expand each retrieved chunk to include its parent context before injecting into the generation prompt. The small child chunk was retrieved precisely. The larger parent chunk provides the surrounding context that the generator needs to produce a coherent answer. This expansion is a post-retrieval operation that occurs before the generation step, adding parent chunk tokens to the context window budget without requiring additional retrieval calls.
05Retrieval pipeline latency budget: advanced RAG retrieval is significantly more expensive than naive RAG retrieval. Query transformation (1 LLM call), multi-query retrieval (3 to 5 vector searches), hybrid search (vector + BM25 in parallel), and re-ranking (cross-encoder over 20 to 50 candidates) together add 200 to 600ms to the retrieval step before generation begins. Teams that implement all advanced RAG optimizations simultaneously without measuring their latency contributions may create a system that is too slow for the application's response time requirements. Measuring each component's latency contribution is as important as measuring its quality contribution.

Advanced RAG retrieval's most important design principle is that each optimization should be measured independently against a controlled baseline before being added to the production pipeline. HyDE helps some query types and hurts others — queries that are already specific and well-formed benefit less from HyDE than queries that are vague or short. Re-ranking helps when the initial retrieval recall is high enough that the correct documents are in the top-20 or top-50 candidate set; it cannot help when the correct documents are not retrieved at all. Knowing which problem each optimization solves prevents adding complexity that does not improve quality for the specific query distribution the application handles.

Composable Modules

Modular RAG

Adaptive Multi-Source Retrieval

Retrieval Pattern

Query classification and routing

Parallel multi-backend search

Graph expansion of initial results

Cross-backend result fusion

Per-module latency monitoring

01Query classification before retrieval determines which module chain and which index combinations run for each incoming query. A lightweight classifier (either a small trained model or a few-shot LLM prompt) categorizes the query by type — factual, analytical, conversational, time-sensitive, cross-domain — and selects the appropriate retrieval path. Queries classified as factual route to a single precise retrieval path. Queries classified as analytical route through multi-query expansion and result fusion. Time-sensitive queries route to temporally filtered indexes. The routing overhead (20 to 50ms) is recovered in reduced downstream retrieval cost for queries that do not need the full module chain.
02Parallel multi-backend retrieval: for queries routed to multiple retrieval sources, the retrieval module fires all relevant backend queries simultaneously rather than sequentially. Dense vector search, sparse keyword search, graph database traversal, and structured SQL queries run in parallel, with the fusion module collecting all results as they complete. Parallel execution bounds the retrieval latency to the slowest backend rather than the sum of all backends. If the slowest backend takes 150ms and four others take under 100ms, the total multi-backend retrieval latency is 150ms rather than 600ms for sequential execution.
03Graph expansion after semantic seed retrieval: the semantic search step identifies the most directly relevant documents. The graph traversal step expands the result set by following citation links, version relationships, and topical co-occurrence edges from the seed documents. Documents that are not semantically similar to the query but are structurally connected to semantically similar documents surface through graph expansion. This two-phase retrieval handles the class of questions where the answer requires understanding relationships between documents rather than the content of any single document in isolation.
04Cross-backend result fusion: combining results from heterogeneous backends (vector store, BM25 index, graph database, SQL database) requires a fusion strategy that normalizes relevance signals from different retrieval systems into a comparable ranking. Reciprocal rank fusion is backend-agnostic and works well across all source types. Score-based fusion requires normalizing scores from different systems to a common scale, which is backend-specific. The fusion module must handle the case where some backends return no results for a given query without failing the entire retrieval operation for the queries where other backends did return relevant content.
05Per-module latency and quality monitoring: modular RAG systems should track retrieval performance at the module level, not just at the overall pipeline level. Knowing that query classification adds 30ms, dense retrieval adds 80ms, graph expansion adds 120ms, and fusion adds 20ms is more useful for optimisation than knowing that the total retrieval step takes 250ms. Per-module monitoring enables targeted optimisation — replace only the slowest module, or cache the most expensive module's results for repeated similar queries — rather than requiring the entire pipeline to be re-architected when latency requirements tighten.

Modular RAG retrieval's distinguishing property is that different queries activate different retrieval paths, producing consistently better retrieval precision per query than a one-size-fits-all approach. The precision improvement comes at the cost of routing accuracy — if the query classifier misclassifies a query, it routes it to an inappropriate module chain that produces worse retrieval than the simplest approach would. Modular RAG quality is bounded above by the quality of the optimal module chain for each query type, and bounded below by the quality of the worst misclassification the router makes in production. Monitoring router accuracy in production is as important as monitoring any individual module's quality.

Model-Controlled Retrieval

Agentic RAG

Iterative Tool-Calling Retrieval

Retrieval Pattern

Agent decides whether to retrieve

Agent formulates retrieval queries

Tool-calling retrieval

Multi-round iterative retrieval

Self-evaluation of retrieved content

01Retrieval necessity decision: before calling any retrieval tool, the agent evaluates whether the current question requires external knowledge or can be answered from its existing context. Simple follow-up questions in an ongoing conversation, questions about the current session's established context, and questions within the agent's training knowledge may not require retrieval at all. This retrieval-when-necessary behavior reduces unnecessary retrieval latency and context window overhead for queries that do not benefit from additional retrieved content.
02Agent-formulated retrieval queries: the agent constructs the retrieval query based on its understanding of what information is needed to answer the question, not just based on the literal user query text. For a complex question, the agent may decompose it into several sub-questions and retrieve for each separately. For a question requiring background context before the specific answer can be understood, the agent retrieves background information first. The retrieval query formulated by the agent is typically more precise and more targeted than the original user question — which is the primary quality advantage of agentic retrieval over fixed-pipeline retrieval.
03Iterative multi-round retrieval: after reviewing first-round retrieval results, the agent evaluates whether it has sufficient information to answer the question or whether gaps remain. If gaps are identified, the agent formulates a new retrieval query targeting the specific missing information and retrieves again. This iterative loop continues until the agent judges that it has sufficient grounding context for a confident answer or until a configured maximum round count is reached. The iterative loop is what enables answering complex multi-hop questions that require assembling information from multiple retrieval rounds in a deliberate sequence.
04Retrieved content evaluation: after each retrieval round, the agent explicitly assesses whether the returned documents are relevant to the question and whether they actually support the type of answer the question requires. Documents that are topically adjacent but do not contain the specific information needed are excluded from the generation context. This in-loop relevance filtering reduces the noise in the final context window — removing documents that are similar enough to the query to be retrieved but do not actually contain useful information for the answer. Fixed pipeline RAG has no equivalent mechanism for in-loop relevance filtering.
05Retrieval trace as an audit log: every tool call made during the retrieval loop appears in the agent's reasoning trace with the query formulated, the tool called, and the results received. This trace provides an audit log of every retrieval decision the agent made before producing the answer. For regulated industries where answer provenance must be auditable — healthcare, legal, financial services — the retrieval trace provides the documentation of which sources were consulted and which were used in producing the final answer. No fixed pipeline RAG architecture produces an equivalent decision-level audit trail as a natural output of the retrieval process.

Agentic RAG retrieval quality is bounded by the model's ability to formulate good retrieval queries. A model that formulates vague, ambiguous retrieval queries benefits less from agentic control than a model that formulates precise, targeted ones. This creates a quality dependency that does not exist in fixed pipeline RAG: the retrieval quality scales with the model's reasoning capability rather than with the retrieval algorithm's configuration. Upgrading the underlying model improves agentic RAG retrieval quality automatically. Changing the model in fixed pipeline RAG requires re-tuning the retrieval algorithm configuration if the new model produces different embedding characteristics.

04

How does each approach integrate retrieved context with generation?

Retrieval delivers source material. Augmentation determines how that material enters the model's context and how the model is instructed to use it. The same retrieved documents can produce very different generation quality depending on how they are formatted, ordered, compressed, and framed in the generation prompt.

View

The Baseline

Naive RAG

Direct Injection, Fixed Template

Augmentation Pattern

Concatenate all top-k chunks

Fixed prompt template

No compression

No relevance filtering

Single generation pass

01Direct injection: concatenate all top-k retrieved chunks into a single context block, prepend the user's original query and a simple instruction ("Answer the question using only the following context"), and pass the complete prompt to the LLM. The model receives everything retrieved without any filtering, compression, or relevance assessment. This approach is maximally simple and maximally transparent — there is no post-retrieval processing that could introduce additional errors between retrieval and generation.
02Fixed prompt template: the instruction framing for how the model should use retrieved context is the same for every query. "Use the following context to answer the question. If the answer is not in the context, say you don't know." This single template is the primary instruction for the model's generation behavior. The template does not adapt to query type (factual versus analytical), answer format requirements (sentence versus list versus table), or domain constraints (cite sources, express uncertainty, use specific terminology).
03Context window consumption: with k=5 and 512-token chunks, naive RAG injects 2,560 tokens of retrieved context into every generation call regardless of the query's complexity. Simple queries that require only one relevant chunk to answer still receive all five chunks. Complex queries that might benefit from more than five chunks are artificially limited to the configured k. The fixed k creates an implicit trade-off that is not calibrated to the specific query: under-context for complex queries, over-context for simple ones.
04No post-generation verification: once the LLM produces an answer, the naive RAG pipeline returns it to the user without verifying that the answer is actually grounded in the retrieved context. The model may produce a hallucination despite receiving relevant context — either by ignoring the context or by generating content that goes beyond what the context supports. No mechanism exists to detect this failure before the answer reaches the user.
05Chunk ordering in the context block: naive RAG typically injects chunks in retrieval score order — highest similarity first. Research on LLM attention patterns (including the lost-in-the-middle findings) suggests that placing the most relevant chunk first or last rather than in the middle of the context block improves the model's tendency to ground its answer in that chunk. Even without changing the retrieval algorithm, reordering the injected chunks so the highest-relevance chunk appears last (exploiting the recency effect) produces measurable quality improvements on some query types at no additional cost.

Naive RAG augmentation treats the generation step as a black box that consumes retrieved context and produces answers. The only control knobs are the prompt template and the order of injected chunks. Both of these are underutilised in most naive RAG implementations — teams write a single generic template at setup time and never revisit it, and inject chunks in default similarity order without evaluating whether a different ordering produces better answers. Before investing in advanced RAG optimizations, teams should first measure whether better prompt template design and chunk ordering improve answer quality on their specific query distribution.

Pre and Post-Retrieval Optimization

Advanced RAG

Compressed, Reordered, Verified

Augmentation Pattern

Context compression (LLMLingua)

Relevance-ordered injection

Query-adaptive templates

Source citation enforcement

Faithfulness verification

01Context compression before injection: apply a compression model that extracts only the sentences most relevant to the current query from each retrieved chunk before injecting into the generation prompt. Tools such as LLMLingua achieve 3 to 5 times compression ratios while preserving the key information needed for answer generation. The compressed context occupies less of the context window, leaving more budget for the model's reasoning and reducing the attention dilution that occurs when the model must attend to large amounts of loosely relevant text before finding the key information.
02Relevance-ordered injection: place chunks in the generation context ordered by their relevance scores from the re-ranking step, with the highest-relevance chunk at the end of the context block (exploiting the recency effect documented in LLM attention research). If three chunks are injected, the most relevant appears last, the second-most-relevant appears first, and the third appears in the middle. This ordering is counterintuitive — placing the most important information at the end rather than the beginning — but is empirically validated to improve answer quality on tasks where the model must draw on the most relevant source above all others.
03Query-adaptive prompt templates: maintain a library of prompt templates optimized for different query types and switch between them based on query classification. Factual queries receive a concise, direct-answer template. Analytical queries receive a template that instructs structured reasoning. Multi-part queries receive a template that encourages addressing each component explicitly. Template selection is either rules-based (query contains a comparison keyword, use the comparison template) or model-based (classify the query then select the template). Query-adaptive templates produce measurably better answer format adherence than a single generic template across diverse query types.
04Source citation enforcement: advanced RAG prompt templates often include explicit instructions to cite the source document or chunk for each claim in the generated answer. This citation enforcement serves two purposes: it improves the model's tendency to stay grounded in retrieved content (generating a citation forces the model to identify which source supports each claim), and it enables downstream verification of the model's sourcing claims. Citation quality varies significantly across models — some models reliably generate accurate citations while others confabulate sources.
05Faithfulness verification after generation: after the LLM produces an answer, pass the answer and the retrieved context to a secondary faithfulness check — either a separate LLM call that evaluates whether each claim in the answer is supported by the retrieved context, or an NLI (Natural Language Inference) model that scores the entailment relationship between the answer and the source documents. Answers that fail the faithfulness check are either regenerated with additional constraints, returned to the user with an explicit uncertainty flag, or escalated to a human reviewer. The additional verification step adds latency but reduces the hallucination rate for high-stakes applications.

Advanced RAG augmentation optimizations are most valuable when the baseline naive RAG generation quality is limited specifically by how the retrieved context is presented to the model rather than by the quality of what was retrieved. If retrieval quality is low (the correct documents are not being retrieved), augmentation improvements produce little benefit. If retrieval quality is high but the model is not properly utilizing the retrieved context, augmentation improvements — compression, reordering, better templates, faithfulness verification — can produce significant quality improvements. Diagnosing whether the failure is in retrieval or augmentation before investing in either is the most important evaluation step.

Composable Modules

Modular RAG

Pipeline-Configured Augmentation

Augmentation Pattern

Per-query augmentation path

Format-aware context assembly

Multi-source attribution

Conditional generation strategies

Generation module swapping

01Per-query augmentation path: the same routing logic that determines which retrieval modules run also determines which augmentation approach is applied after retrieval. Factual queries use a concise direct-answer augmentation template. Queries requiring synthesis across multiple sources use a structured-reasoning augmentation template that explicitly instructs the model to compare and reconcile multiple source perspectives. Long-form generation queries use an outline-first augmentation template that generates a plan before executing the full response. The augmentation strategy adapts to the query type rather than applying one format universally.
02Format-aware context assembly: modular RAG systems that retrieve from multiple backend types — structured SQL results, vector-retrieved text chunks, graph traversal summaries — must assemble heterogeneous content into a coherent generation context. Each content type requires different formatting: SQL query results formatted as tables, text chunks formatted as prose with source labels, graph traversal results formatted as relationship summaries. The context assembly module handles this format normalization before the generation module receives the assembled context.
03Multi-source attribution: when content is retrieved from multiple distinct sources simultaneously, the generation prompt must communicate clearly which content came from which source so the model can attribute claims correctly in its answer. Modular RAG augmentation templates use explicit source labeling — [Source: Policy Manual, Section 4.2] or [Source: Support Ticket #TK-2024-8821] — to enable the model to generate properly attributed answers that users can verify against the originating sources.
04Conditional generation strategies: some query types benefit from generation strategies that are not available in fixed pipeline RAG. Queries with high uncertainty about the correct answer benefit from a calibrated uncertainty generation strategy that expresses confidence levels explicitly rather than producing a single confident answer. Queries where the retrieved context is insufficient to produce a full answer benefit from a partial-answer strategy that generates what is known from the retrieved context and identifies what additional information would be needed. Modular RAG can implement these conditional strategies as distinct generation module configurations rather than baking them into a single universal template.
05Generation module swapping by cost and quality requirements: modular RAG can route different query types to different LLM models for the generation step. Simple factual queries that require minimal reasoning route to a smaller, faster, cheaper model. Complex analytical queries that require multi-step reasoning route to a larger, more capable model. This tiered generation routing reduces inference cost for the majority of queries (which are simple) while maintaining quality for the minority of queries (which are complex) without requiring the application to choose one model that must serve all query types equally.

Modular RAG augmentation's primary advantage over fixed pipeline approaches is that the generation strategy can be as varied as the query types the application handles. A single generation template is an implicit assumption that all queries benefit from the same instructional framing, which is false for any application with diverse query types. The value of query-adaptive augmentation templates scales with query diversity: applications with one dominant query type gain little from adaptive templates; applications with many distinct query types gain significantly from matching each type to its optimal augmentation strategy.

Model-Controlled Retrieval

Agentic RAG

Agent-Assembled Context, Self-Verified Output

Augmentation Pattern

Agent selects what to include

Incremental context building

Self-RAG reflection tokens

Grounded generation with citations

Agent decides when answer is complete

01Agent-assembled context: rather than having an infrastructure layer assemble the generation context and pass it to the model, the agent itself decides what retrieved content to include in its final answer context. After completing its retrieval rounds, the agent reviews all retrieved content and selects the specific documents and passages that are most relevant to the question. This agent-level curation of the final generation context is more precise than any automated filtering step because the agent reasons about relevance in the context of the specific question it is trying to answer rather than using a generic relevance heuristic.
02Incremental context building across retrieval rounds: the agent's context window accumulates retrieved information progressively across multiple retrieval rounds. After the first retrieval round, the agent has context C1. After evaluating C1 and retrieving again, the agent has context C1 plus C2. The agent's reasoning at each step is informed by all previously retrieved content rather than just the current round's results. This accumulated context enables reasoning about how different retrieved documents relate to each other — identifying contradictions, identifying gaps, identifying which source is most authoritative on the specific point at issue.
03Self-RAG reflection before final answer generation: before committing to a final answer, the agent explicitly evaluates three things: whether it has retrieved sufficient context to answer the question (retrieval adequacy), whether the retrieved documents actually support the answer it is about to generate (document support), and whether the answer it plans to generate is fully grounded in the retrieved context (generation faithfulness). These explicit self-evaluations before final answer generation reduce hallucination rates in domains where grounded, accurate answers are more important than fluent, confident-sounding ones.
04Grounded generation with inline citations: agentic RAG agents naturally produce inline citations as part of their reasoning process — the agent references specific retrieved documents when making claims because it has direct access to those documents in its context window from its own retrieval tool calls. This natural citation behavior (as distinct from post-hoc citation generation) tends to produce more accurate source attribution because the agent cites documents it actually used in reasoning rather than generating plausible-seeming citations after the fact.
05Completion judgment: the agent decides when it has sufficient grounding to produce a final answer rather than retrieving again. This judgment — "I now have enough context to answer confidently" or "I need one more retrieval round to verify this claim" — is an explicit reasoning step that does not exist in fixed pipeline RAG. The quality of this judgment determines whether the agent terminates retrieval loops too early (producing inadequately grounded answers) or too late (producing expensive over-researched answers to simple questions). Training agents to calibrate this judgment correctly for a specific domain is one of the most productive investments in agentic RAG quality improvement.

Agentic RAG augmentation is the culmination of the architectural progression from naive RAG: the model that generates the answer is also the entity that decided what to retrieve, selected what to include, evaluated the quality of what was found, and judged when the evidence was sufficient. Every fixed pipeline approach separates one or more of these decisions from the generator. Agentic RAG collapses the separation entirely. The quality gain from this integration scales with the model's reasoning capability. The cost is paid in latency and inference expense per query.

05

How is each approach evaluated?

RAG evaluation is harder than model evaluation. A model that produces high MMLU scores may produce poor RAG answers because the retrieval step introduces a failure mode that benchmark tests do not capture. Evaluating RAG requires measuring the retrieval step and the generation step independently, and measuring the compound behavior of both together.

View

The Baseline

Naive RAG

Retrieval Metrics + Answer Metrics

Evaluation Stack

Recall@k (retrieval)

Precision@k (retrieval)

Faithfulness score

Answer relevance

Context relevance

01Retrieval evaluation requires a ground-truth dataset: for each evaluation question, a set of chunks that contain the necessary information to answer it (the relevant chunk set). Recall at k measures what fraction of relevant chunks appear in the top-k retrieved results. Precision at k measures what fraction of the top-k retrieved results are actually relevant. These two metrics trade against each other as k changes: increasing k improves recall but reduces precision. For naive RAG, measuring both with the configured k reveals whether the retrieval step is finding the right content and whether it is finding mostly relevant or mostly irrelevant content.
02Context relevance: given the retrieved context, how relevant is it to the question asked? This can be measured by human annotation (expensive but reliable) or LLM-as-judge evaluation (cheaper, less reliable). A high context relevance score means the retrieval step is consistently returning relevant content. A low context relevance score diagnoses that the retrieval step is the primary quality bottleneck regardless of how good the generation step is.
03Faithfulness: given the retrieved context and the generated answer, to what extent does the generated answer stay grounded in the retrieved content versus generating claims not supported by the context? A high faithfulness score means the model is using the retrieved context reliably. A low faithfulness score means the model is hallucinating over the retrieved context — a failure of the generation step that cannot be fixed by better retrieval. RAGAS and TruLens are the most widely used frameworks for faithfulness evaluation as of 2026.
04Answer relevance: is the generated answer actually responsive to the user's question? A high faithfulness score combined with a low answer relevance score indicates that the model is accurately reproducing retrieved content but the retrieved content is about the right topic without containing the specific answer the question requires. This compound diagnostic — faithful but irrelevant — points to a retrieval quality problem where the correct type of document is being retrieved but the specific chunk containing the answer is not.
05End-to-end evaluation on a question-answer dataset: beyond component-level metrics, evaluate the complete RAG pipeline on a representative set of questions with known correct answers. Exact match (correct answer appears in the generated text), F1 overlap (token-level match between generated and reference answer), and LLM-graded correctness (a separate LLM judges whether the generated answer is semantically equivalent to the reference answer) are the standard end-to-end metrics. End-to-end metrics capture compound failures that component metrics miss: a system can achieve high retrieval recall, high context relevance, and high faithfulness yet still produce wrong answers if the relevant information is retrieved and faithfully reproduced but does not actually answer the question.

Naive RAG evaluation establishes the baseline against which every subsequent architectural investment should be measured. The evaluation dataset and metrics chosen for naive RAG should remain consistent as the system evolves through advanced RAG and modular RAG, because the value of each architectural improvement is its measured delta against the baseline — not its absolute score. Teams that change their evaluation dataset or metrics when they change their architecture lose the ability to attribute quality changes to specific architectural decisions.

Pre and Post-Retrieval Optimization

Advanced RAG

Component Ablation + Latency-Quality Frontier

Evaluation Stack

Component ablation testing

Latency vs quality frontier

Re-ranking MRR improvement

Compression fidelity

A/B testing against naive baseline

01Component ablation testing: for each optimization added to the advanced RAG pipeline, measure the quality impact of that specific component in isolation by adding it to the baseline pipeline and evaluating before adding the next component. The ablation sequence might be: naive RAG baseline → add HyDE, evaluate → add re-ranking, evaluate → add compression, evaluate → add hybrid retrieval, evaluate. Each evaluation step attributes quality improvement or degradation to a specific component, enabling evidence-based decisions about which components to keep and which to remove or tune differently.
02Latency-quality frontier analysis: each advanced RAG optimization improves quality at the cost of additional latency. Plotting the quality-latency trade-off for different component combinations produces a Pareto frontier — the set of configurations where no component can be added that improves quality without also increasing latency. For applications with strict latency SLAs, this frontier identifies the highest quality configuration that fits within the SLA constraint. For applications without latency constraints, it identifies where adding more components produces diminishing quality returns relative to the latency cost.
03Re-ranking improvement measurement: evaluate the Mean Reciprocal Rank (MRR) of the pre-reranking candidate set and the post-reranking final top-k. MRR measures the rank position of the first relevant result: if the most relevant document appears in position 3 of the candidate set, MRR is 1/3. After re-ranking, if the same document moves to position 1, MRR is 1. An MRR improvement from pre to post re-ranking quantifies the value of the re-ranking step independently of the overall end-to-end answer quality, enabling calibration of the re-ranker quality against its latency cost.
04Compression fidelity evaluation: after applying context compression, measure what fraction of the information needed to answer the evaluation questions was preserved in the compressed context versus lost. This is typically evaluated by comparing answers generated from compressed context against answers generated from uncompressed context using the same ground truth. A compression fidelity score above 95% means the compression step loses little useful information. A score below 80% means the compression is removing content needed for correct answers and should be tuned more conservatively.
05A/B testing against naive RAG baseline in production: after validating improvements on an offline evaluation dataset, run the advanced RAG pipeline against a percentage of live production traffic alongside the naive RAG baseline. Compare answer quality (from user feedback signals or automated evaluation), latency, and cost between the two systems. Online A/B testing often reveals quality gaps between offline evaluation performance and production performance because production query distributions differ from evaluation datasets in ways that matter for specific optimization components.

Advanced RAG evaluation requires measuring the contribution of each optimization independently before accepting the compound system into production. The most common evaluation mistake is testing the full advanced RAG system against the naive RAG baseline and attributing the entire quality improvement to all the optimizations collectively. If one of those optimizations is actively harmful for a query type that happens to be underrepresented in the evaluation set, it will degrade production quality while appearing to contribute to the measured improvement. Ablation testing finds these harmful components before they reach production.

Composable Modules

Modular RAG

Per-Module Metrics + Router Accuracy

Evaluation Stack

Router classification accuracy

Per-module quality contribution

Per-query-type evaluation

Module failure impact analysis

Coverage across query types

01Router classification accuracy: the routing module's correctness is the prerequisite for all other modular RAG quality. If the router misclassifies a complex analytical query as a simple factual query and routes it through the simple pipeline, the answer quality will be worse than naive RAG for that query class regardless of how good the complex module chain is. Evaluate router accuracy using a labeled query type dataset: measure precision and recall per query category, and measure the quality of answers produced by misclassified queries. Router errors that route complex queries through simple pipelines are more harmful than router errors in the other direction.
02Per-module quality contribution: measure each module's contribution to the quality of queries that pass through it. If the graph expansion module is activated for 15% of queries and produces measurable quality improvements for 60% of those queries while producing no measurable improvement or degradation for 40%, the module is justified for the 60% at the cost of additional latency for the 40%. If the same module produces no measurable improvement for any query type it is activated for, it should be removed regardless of its theoretical value. Empirical measurement of module contribution overrides architectural intuition.
03Per-query-type evaluation: maintain separate evaluation datasets for each query type handled by the router — factual, analytical, conversational, multi-hop, time-sensitive. Evaluate each module path against its corresponding query type dataset rather than against a single mixed dataset. A mixed evaluation dataset can show strong average quality while masking specific query type failures that matter to users. A factual query module chain scoring well on factual queries but a multi-hop module chain failing on multi-hop queries is a production quality problem that mixed-dataset evaluation may not detect.
04Module failure impact analysis: evaluate what happens to overall system quality when individual modules fail or degrade. If the re-ranking module becomes unavailable (backend outage, latency SLA violation triggering a timeout), does the system gracefully fall back to the pre-ranking result set, or does it fail entirely? Module failure impact analysis identifies the components whose failure has the largest quality impact and justifies investing in redundancy, fallback behavior, and health monitoring specifically for those components.
05Query type coverage analysis: evaluate whether the router's configured query types cover the actual query distribution received in production. Queries that do not match any configured query type either fall into a default category (producing potentially suboptimal routing) or fail to route entirely. Coverage analysis identifies query types that appear frequently in production but are not addressed by any configured module path, providing a prioritized list of new module paths to develop based on actual production query distribution rather than anticipated query types.

Modular RAG evaluation requires more evaluation infrastructure than any other RAG architecture because there are more components to evaluate and more failure modes to detect. Teams that adopt modular RAG without investing in per-module evaluation infrastructure often find that the added complexity produces no measurable quality improvement over advanced RAG — not because modular RAG is ineffective but because without per-module evaluation, misconfigured or underperforming modules go undetected. The evaluation infrastructure is not optional; it is what makes the complexity of modular RAG produce its intended quality benefits.

Model-Controlled Retrieval

Agentic RAG

Trajectory Evaluation + Tool Use Quality

Evaluation Stack

Retrieval trajectory evaluation

Tool call quality assessment

Multi-hop reasoning accuracy

Over-retrieval detection

Completion judgment calibration

01Retrieval trajectory evaluation: agentic RAG produces a sequence of retrieval decisions — each tool call with its query, the results returned, and the agent's subsequent reasoning — that can be evaluated as a trajectory rather than as individual point decisions. A correct answer produced through a suboptimal trajectory (redundant retrieval calls, poorly formulated queries, retrieving from the wrong sources) is a lower quality trajectory than the same correct answer produced efficiently. Trajectory evaluation assesses both the correctness of the final answer and the efficiency and quality of the path taken to reach it.
02Tool call quality assessment: for each retrieval tool call made by the agent, evaluate the quality of the query formulated, the appropriateness of the target knowledge source selected, and the relevance of the results to the specific information gap the agent was trying to fill. Tool call quality assessment identifies failure patterns in how the agent uses its retrieval tools: systematically formulating vague queries, consistently selecting inappropriate knowledge sources for specific question types, or failing to use available filtering parameters that would improve retrieval precision.
03Multi-hop reasoning accuracy: the primary quality advantage of agentic RAG over fixed pipeline approaches is on multi-hop questions requiring iterative information gathering. Evaluate specifically on datasets of multi-hop questions that require connecting information from multiple sources in a deliberate sequence. Strong agentic RAG performance on multi-hop questions while fixed pipeline performance degrades justifies the additional latency and cost of the agentic approach for applications where multi-hop queries are frequent. Agentic RAG that performs similarly to advanced RAG on multi-hop questions is not delivering its theoretical quality advantage and should be investigated for retrieval loop failures.
04Over-retrieval detection: measure the average number of retrieval rounds and total retrieved tokens per query type. An agent that runs 5 retrieval rounds for simple factual questions that can be answered with one round is over-retrieving — wasting latency and cost without improving answer quality. Over-retrieval detection identifies query types where the agent's completion judgment is miscalibrated (it keeps retrieving past the point where additional retrieval improves answer quality) and where targeted prompt improvements or fine-tuning of the completion judgment can significantly reduce per-query cost.
05Completion judgment calibration: evaluate the agent's judgment about when it has sufficient context to produce a final answer versus when additional retrieval is needed. A well-calibrated completion judgment produces confident answers when context is sufficient and additional retrieval rounds when context is genuinely insufficient. Calibration evaluation compares the agent's confidence signals against independently verified answer correctness — a well-calibrated agent should have high correctness when it judges context as sufficient and lower correctness when it continues retrieving (indicating the questions it found difficult are genuinely more difficult). Poor calibration in either direction — over-confidence with insufficient context, or under-confidence leading to excessive retrieval — is a targetable quality improvement.

Agentic RAG evaluation requires evaluating the process, not just the outcome. Two agents that produce the same correct answer with different retrieval trajectories — one using one targeted tool call, one using five unfocused ones — are not equally good systems. The efficient agent will remain efficient at scale; the inefficient one will accumulate latency and cost that degrades user experience and economics under production load. Process evaluation, not just outcome evaluation, is what distinguishes agentic RAG systems that are production-ready from those that are research prototypes producing correct answers through inefficient means.

06

What are the production trade-offs of each approach?

Every RAG architecture that works in development faces a different version of itself in production: more queries, more diverse queries, more failure modes, more operational complexity, and more cost scrutiny. The architecture that is best in a prototype is not always the one that is best at the scale and operational complexity of production.

View

The Baseline

Naive RAG

Low Ops Cost, Predictable Failure, Quality Ceiling

Production Profile

Minimal ops complexity

Predictable latency

Easy to debug

Quality ceiling at scale

Best starting architecture

01Operational simplicity is naive RAG's most durable production advantage. Two services — a vector database and an LLM API — cover the entire infrastructure requirement. Failures are easy to diagnose: vector database down, retrieval fails; LLM API down, generation fails. There are no intermediate services, no routing components, no module chains, no consolidation processes. Teams that underestimate the operational cost of more complex RAG architectures often return to naive RAG for subsystems where simplicity is more valuable than quality headroom.
02Predictable latency: every query follows the same pipeline — one embedding call, one vector search, one generation call. The latency distribution is narrow and predictable. P50 and P99 latency are close together because there are no variable components (routing, iterative retrieval, multi-round agent loops) that produce wide latency distributions. For applications with strict SLAs, the predictability of naive RAG latency is sometimes more valuable than the quality improvements of more complex architectures with wider latency distributions.
03Debugging ease: when naive RAG produces a wrong answer, there are exactly two places to look — retrieval quality (did we retrieve the right chunks?) and generation quality (did the model use the retrieved context correctly?). Each of these is inspectable by looking at the vector search results and the LLM prompt respectively. In more complex architectures, wrong answers may originate from any of many components with many potential interaction effects. The debuggability advantage of naive RAG makes it the correct choice for teams where engineering time for debugging is the binding constraint.
04Quality ceiling at scale: as the document collection grows and query diversity increases, naive RAG's fixed chunk size, single-pass retrieval, and lack of query adaptation produce a quality ceiling that more sophisticated approaches can exceed. For applications serving one focused use case with a stable, moderate-sized document collection, the ceiling may never be reached. For applications serving diverse use cases with large, diverse document collections and complex queries, the ceiling is reached quickly. Recognizing when naive RAG has reached its quality ceiling — and not before — is the signal to invest in more complex architectures.
05Cost profile: naive RAG cost per query is the sum of one embedding call and one generation call. There are no additional LLM calls for query transformation, re-ranking, compression, or faithfulness verification. For high-volume applications where cost per query is a significant budget item, naive RAG's minimal per-query cost is a financial advantage over more complex approaches. The lower cost can be invested in a better embedding model, a more capable generator, or a larger context window allocation — all of which improve naive RAG quality within its architectural constraints.

Naive RAG is the right production architecture for every application at launch, and the right architecture for some applications indefinitely. The principle is not that naive RAG is inadequate — it is that it has a measurable quality ceiling that becomes visible at specific scale and query complexity thresholds. Reaching that ceiling is the trigger for architectural investment, not the expectation that the ceiling will be reached. Teams that spend months building advanced or modular RAG before knowing whether naive RAG's quality is sufficient for their specific application are investing engineering time without evidence that the investment is needed.

Pre and Post-Retrieval Optimization

Advanced RAG

Quality Gains, Latency Costs, Tuning Burden

Production Profile

15-30% quality improvement typical

200-600ms additional latency

Higher per-query LLM cost

Re-ranker maintenance

Component version management

01Quality improvement range: advanced RAG optimizations collectively produce 15 to 30% improvement in end-to-end answer quality over naive RAG on representative evaluation datasets for most production applications. The improvement is not uniform across query types — factual precision-recall improvements from re-ranking are larger than improvements on already-well-served simple factual queries. The 15 to 30% range reflects typical production deployments; specific applications may see higher improvements if naive RAG was severely limited by one of the problems advanced RAG addresses, or lower improvements if naive RAG was already performing near its quality ceiling for that specific query type distribution.
02Latency cost of the optimization stack: each advanced RAG component adds latency. Query transformation via HyDE (one LLM call, 100 to 300ms), multi-query retrieval (parallel vector searches, 50 to 100ms overhead), re-ranking (cross-encoder over top-20 candidates, 100 to 200ms), and context compression (one LLM call or fast compression model, 50 to 150ms) together add 300 to 750ms to the naive RAG baseline latency before the final generation step begins. Applications with sub-second total response time SLAs must carefully select which components fit within the SLA rather than implementing all available optimizations.
03Per-query cost increase: advanced RAG makes additional LLM calls for query transformation and context compression that are not present in naive RAG. At high query volumes, these additional calls represent significant additional inference cost. At 1 million queries per month, adding one additional LLM call per query at $0.001 per call adds $1,000 per month in inference cost. For applications already at scale, the cost impact of each additional LLM call should be budgeted before adding query transformation or verification components.
04Re-ranker maintenance: cross-encoder re-ranking models require separate deployment and maintenance alongside the primary LLM. The re-ranker model version must be monitored — outdated re-ranker models may produce lower-quality relevance scores as the document collection evolves. Re-ranker latency contributes to the overall retrieval pipeline latency and must be monitored independently. At production scale, the re-ranker becomes a critical path service whose availability determines whether the advanced RAG quality improvement is actually delivered or falls back to naive RAG baseline quality.
05Component version management: advanced RAG systems are composed of multiple models — the embedding model, the query transformation LLM, the re-ranker, the context compression model, and the generator. When any component model is updated (new embedding model version, new re-ranker checkpoint, new generator), the impact on end-to-end quality must be re-evaluated because component updates can interact with each other in unexpected ways. A new embedding model may change the embedding space in ways that require re-tuning the re-ranker's relevance threshold. Managing the evaluation implications of component version updates is an ongoing operational responsibility unique to multi-component advanced RAG systems.

Advanced RAG is the right architecture for the majority of production applications that have outgrown naive RAG's quality ceiling. The 15 to 30% quality improvement is real and consistent across diverse applications. The challenge is selecting the right subset of optimizations for each specific application based on where naive RAG specifically fails, rather than implementing all optimizations and assuming the full quality improvement is available for all query types. Most advanced RAG deployments benefit most from two or three well-chosen optimizations rather than from the full optimization stack.

Composable Modules

Modular RAG

Highest Flexibility, Highest Ops Complexity

Production Profile

Best per-query-type quality

High infrastructure complexity

Router accuracy as quality floor

Multiple failure surfaces

Justified for high query diversity

01Per-query-type quality advantage: modular RAG produces better quality than advanced RAG for each specific query type when the module chain is correctly configured for that type, because it applies the optimal combination of optimizations for each query rather than a fixed combination optimized for average performance. For applications with very diverse query types — customer service agents answering product questions, billing questions, technical support questions, and account management questions simultaneously — the per-type quality optimization produces a meaningful quality advantage over fixed pipeline approaches.
02Infrastructure complexity: modular RAG requires managing multiple retrieval backends (vector store, BM25 index, graph database), a routing component, a fusion service, potentially multiple generator configurations, and the monitoring and evaluation infrastructure for each. The operational complexity is proportional to the number of modules and the number of distinct module paths. Teams should count the distinct services that require individual uptime monitoring, individual capacity planning, and individual version management before committing to modular RAG. The engineering team size and operational maturity needed to run modular RAG reliably is higher than for any other approach in this comparison.
03Router accuracy as the quality floor: the routing module's accuracy sets the minimum quality the system delivers across all queries. A router that correctly classifies 80% of queries and misclassifies 20% delivers the optimal module chain quality for the 80% and the wrong module chain quality for the 20%. If the 20% of misclassified queries are the most important queries the application handles, the router's quality floor creates a significant production quality problem. Router accuracy should be measured and monitored as a first-order production metric rather than as an implementation detail of the retrieval pipeline.
04Multiple independent failure surfaces: each module is a potential failure point. A degraded re-ranking service, a BM25 index update failure, a graph database availability issue, or a router classification model drift can each independently reduce answer quality for specific query types without taking the entire system down. Failures in modular RAG are often partial — some query types are affected while others continue to function normally. This partial failure mode requires more sophisticated production monitoring that tracks quality per query type rather than just overall system health, because an overall-healthy system may have one query type category that is silently failing.
05Cost efficiency through adaptive routing: modular RAG's routing-based optimization reduces per-query cost for simple queries by routing them through lightweight module chains, which partially offsets the infrastructure cost of maintaining multiple module paths. If 70% of queries are simple factual queries that route through a single inexpensive retrieval call and a small generator model, the cost savings on that 70% of traffic can fund the additional infrastructure cost for the 30% of complex queries that require the full module stack. The net cost of modular RAG versus advanced RAG depends on the query complexity distribution — applications with mostly simple queries may find modular RAG cheaper than advanced RAG despite higher infrastructure complexity.

Modular RAG is justified when two conditions are simultaneously true: the application serves genuinely diverse query types that would each benefit from different retrieval strategies, and the engineering team has the operational maturity to run multi-service infrastructure with module-level monitoring and evaluation. Meeting only the first condition without the second produces a system that is architecturally capable of higher quality but operationally unable to maintain it. Meeting only the second condition without the first produces operational complexity that does not translate into measurable quality improvement over advanced RAG.

Model-Controlled Retrieval

Agentic RAG

Best for Complex Queries, Worst Latency, Highest Cost

Production Profile

Superior on multi-hop queries

Variable latency (1-10+ retrieval rounds)

Highest per-query LLM cost

Transparent retrieval decisions

Scales with model quality

01Quality advantage on complex queries: agentic RAG produces the highest answer quality for multi-hop questions, questions requiring synthesis across many distinct sources, and questions where the necessary retrieval strategy cannot be determined in advance from the question text alone. For applications where these complex query types are the core use case — research assistants, due diligence tools, technical support for complex systems — agentic RAG's quality advantage over fixed pipeline approaches is significant and consistent. For applications where the majority of queries are factual and self-contained, agentic RAG's quality advantage is marginal and does not justify the additional latency and cost.
02Variable latency is the most significant production challenge for agentic RAG. A query requiring one retrieval round completes in roughly the same time as advanced RAG. A query requiring five retrieval rounds takes five times the retrieval latency plus the agent's reasoning time between rounds. The P99 latency for agentic RAG is dramatically higher than the P50 latency because the most complex queries produce the most retrieval rounds. Applications with strict P99 latency SLAs must either implement maximum round limits (capping quality to limit latency) or segment traffic so complex queries are served through agentic RAG without affecting latency commitments for the majority of simpler queries.
03Highest per-query LLM cost: agentic RAG makes multiple LLM calls per query — each reasoning step between retrieval rounds, the self-evaluation steps, and the final generation. At 3 to 5 retrieval rounds with one LLM call per round plus the final generation, the total LLM calls per complex query is 4 to 6 times higher than naive RAG. For high-volume applications, this cost multiplier is significant. Most production agentic RAG deployments implement query complexity classification at the entry point, routing simple queries to naive or advanced RAG and reserving agentic RAG for the subset of queries where iterative reasoning is genuinely necessary.
04Retrieval decision transparency as a compliance asset: the agentic RAG reasoning trace documenting every retrieval decision made before producing an answer is a compliance asset in regulated industries. Healthcare AI applications that must document which information sources were consulted before a clinical recommendation, legal AI applications that must cite the specific documents supporting a legal analysis, and financial AI applications that must demonstrate which data points informed a recommendation all benefit from the natural audit trail produced by agentic RAG's tool-calling trace. No fixed pipeline approach produces an equivalent decision-level documentation automatically.
05Quality scaling with model capability: agentic RAG quality improves as the underlying model improves, because better models formulate better retrieval queries, make better completion judgments, and synthesize retrieved content more accurately. Fixed pipeline RAG quality is less sensitive to model improvements because the retrieval strategy is fixed and the quality ceiling is set by the pipeline configuration rather than by the model's reasoning capability. For applications committed to staying at the frontier of model capability, agentic RAG architecture provides a better return on model quality improvements than fixed pipeline architectures.

Agentic RAG is the architecture that produces the highest quality answers for the most complex queries, at the highest cost and the highest latency. The correct production decision is not whether agentic RAG is better than advanced RAG in absolute terms — it usually is for complex queries — but whether the application's query complexity distribution, latency tolerance, and budget justify the architectural investment. For applications where 10% of queries are genuinely complex and 90% are simple, a hybrid architecture that applies agentic RAG to the 10% and advanced RAG to the 90% typically produces better overall system economics than applying either approach uniformly to all queries.

M

Methodology

This is the final volume in the AI Systems Studies series. The methodology for this volume and the principles that governed all eight volumes.

📄

Technical Literature

RAG Survey (Gao et al. 2023), Self-RAG (Asai et al. 2023), HyDE (Gao et al. 2022), RAGAS evaluation framework (Es et al. 2023), and production engineering reports from leading AI teams. Performance ranges cited are from benchmarks on representative open-domain QA datasets. Specific application performance varies significantly.

⚙️

Production Reports

Engineering blog posts, conference talks (NeurIPS 2025, MLSys 2026, ACL 2026) and published case studies from organisations deploying RAG at production scale. All production claims represent observed patterns across multiple deployments, not guarantees for any specific deployment configuration.

🔬

Benchmark Data

Recall and precision improvement ranges from independent re-ranking evaluations on standard RAG benchmarks (BEIR, MS MARCO). Latency ranges from measured production deployments with standard cloud infrastructure. All quantitative values are directional. Specific values depend heavily on model versions, hardware and query distributions.

💭

Series Principles

Across all eight volumes: data from official sources, honest documentation of limitations and maintenance mode status, no hype, no guaranteed recommendations. The series compares what is true rather than what performs best in marketing materials. All insights are reasoned conclusions, not facts.

On the series as a whole

Eight volumes. Eight areas of AI infrastructure. The series started with production AI architecture at the company level and ended with RAG at the algorithm level. The series did not follow a predetermined plan. Each volume was written after enough research to determine which topics genuinely warranted the depth of coverage the format provides.

Two patterns appeared in the research across volumes that were not anticipated at the start: the frequency with which incumbent tools entered maintenance mode during the research window (TGI, Helicone) and the degree to which the most important architectural decisions are the ones that are hardest to see: write policy in memory systems, filtering in vector databases, chunking in RAG, context position in prompt engineering. The less visible decision consistently matters more than the more visible one.

Research timeline

Researched October 2026. Primary sources for this volume: RAG survey papers, Self-RAG paper, HyDE paper, RAGAS and TruLens evaluation framework documentation, LlamaIndex and LangChain RAG documentation, production engineering reports from Databricks, Cohere and Anthropic on RAG deployment at scale. Secondary sources: independent RAG benchmark comparisons published through September 2026.

Last Updated: October 18, 2026

Swarnim
Tiwari

AI Systems Researcher

This is the last volume in the series. Eight months of research, eight areas of AI infrastructure, eight attempts to explain something complex clearly enough that someone else can use the understanding.

The thing I did not anticipate when I started: the most important engineering decisions in AI systems are consistently the invisible ones. Not which model to use but how to chunk the documents it reads. Not which vector database to buy but what metadata to store alongside the embeddings. Not which RAG architecture to adopt but what to write to memory and when. The glamorous decisions are usually not the ones that determine whether the system actually works.

RAG felt like the right place to end the series because it connects most of the topics covered before it: vector databases (Vol 03), context engineering (Vol 06), memory systems (Vol 07) and inference infrastructure (Vol 05) are all part of a complete RAG deployment. The series was, in retrospect, building toward this.

I am a student in India. This series started because I was reading about AI and learning nothing useful. I built it for the next person who has that experience.

LinkedIn GitHub

AI Systems Studies — Complete Series

Vol. 01Production AI Architecture = OpenAI, Anthropic, Palantir, NVIDIAPublished

Vol. 02AI Agent Frameworks = OpenAI SDK, LangGraph, CrewAI, MastraPublished

Vol. 03Vector Databases = Pinecone, Weaviate, Milvus, QdrantPublished

Vol. 04AI Observability = LangSmith, Langfuse, Helicone, W&B WeavePublished

Vol. 05Inference Infrastructure = vLLM, SGLang, TensorRT-LLM, TGIPublished

Vol. 06Context Engineering = Mem0, Zep, LangMem, LettaPublished

Vol. 07Memory Systems = In-Context, External, Episodic, SemanticPublished

Vol. 08RAG Architectures = Naive, Advanced, Modular, AgenticThis Study