RAPTOR, HyDE, and RAG Fusion are three advanced retrieval techniques that each address a different fundamental limitation of standard vector RAG. This guide gives each the depth it deserves: the core idea, why it works, when to use it, and a working implementation you can drop into an existing pipeline.
RAPTOR: Retrieval at Multiple Levels of Abstraction
Standard RAG retrieves at the chunk level: individual passages. RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval) builds a tree of summaries — clusters of chunks are summarised by an LLM, those summaries are clustered and summarised again, repeating until the entire corpus is represented by a small number of high-level summaries. At query time, RAPTOR retrieves from all levels of the tree simultaneously. A specific factual question finds its answer in a leaf chunk. A broad thematic question (“what are the main arguments in this corpus?”) finds relevant material in the high-level summary nodes that capture themes across many documents.
The key insight: embedding space clusters semantically related content, and summarising those clusters produces distilled representations of topics that are impossible to retrieve with leaf-level chunks. A query about “the overall strategic direction” will not find a single chunk that answers it directly, but a RAPTOR summary node synthesising dozens of strategy-related chunks will match it well.
Implementing RAPTOR: The Core Algorithm
import numpy as np
from sklearn.mixture import GaussianMixture
import ollama
def embed(text: str) -> list[float]:
return ollama.embeddings(model="nomic-embed-text", prompt=text)["embedding"]
def cluster_chunks(chunks: list[str], n_clusters: int = None) -> list[list[str]]:
"""Cluster chunks by semantic similarity using GMM."""
embeddings = np.array([embed(c) for c in chunks])
# Auto-select cluster count if not specified
if n_clusters is None:
n_clusters = max(2, len(chunks) // 10) # ~10 chunks per cluster
gm = GaussianMixture(n_components=n_clusters, random_state=42)
labels = gm.fit_predict(embeddings)
clusters = [[] for _ in range(n_clusters)]
for chunk, label in zip(chunks, labels):
clusters[label].append(chunk)
return # remove empty clusters
def summarise_cluster(chunks: list[str]) -> str:
"""Summarise a cluster of chunks into a single representative summary."""
combined = "\n\n---\n\n".join(chunks[:10]) # limit to 10 chunks
response = ollama.chat(
model="llama3.1",
messages=[{"role": "user", "content":
f"Summarise the key information in these passages into a single coherent paragraph:\n\n{combined}"
}],
options={"temperature": 0.2, "num_predict": 300}
)
return response["message"]["content"]
def build_raptor_tree(leaf_chunks: list[str], max_levels: int = 3) -> list[list[str]]:
"""Build multi-level summary tree. Returns all levels for indexing."""
all_levels = [leaf_chunks]
current_level = leaf_chunks
for level in range(max_levels):
if len(current_level) <= 5: # stop when too few chunks to cluster
break
clusters = cluster_chunks(current_level)
summaries = [summarise_cluster(cluster) for cluster in clusters]
all_levels.append(summaries)
current_level = summaries
print(f"Level {level+1}: {len(clusters)} clusters -> {len(summaries)} summaries")
return all_levels
# Build and index the tree
all_levels = build_raptor_tree(leaf_chunks)
# Index all levels together
all_texts = []
for level_idx, level_chunks in enumerate(all_levels):
for chunk_idx, chunk in enumerate(level_chunks):
all_texts.append({
"text": chunk,
"level": level_idx,
"id": f"level{level_idx}_chunk{chunk_idx}"
})
# Standard ChromaDB indexing of all_texts...
# Retrieval is identical to standard RAG — RAPTOR just enriches the indexRAPTOR’s retrieval is identical to standard RAG — you query the combined index and return the most similar chunks, which may be leaf-level passages or summary nodes depending on what best matches the query. The tree structure enriches what is available to retrieve, not how retrieval works. The main cost is the LLM summarisation during indexing: a corpus of 1000 chunks requires roughly 100 LLM summarisation calls at the first level, 10 at the second, and 1-2 at the third. This is manageable with local Ollama models (running overnight for large corpora) and is a one-time cost that pays dividends on every subsequent query.
When RAPTOR Helps Most
RAPTOR provides the largest quality improvement on broad synthesis questions that no single chunk can answer. Research corpora where users ask “what does the literature say about X?”, internal document collections with varied document types where themes span multiple files, and knowledge bases with strong topic clustering all benefit substantially. RAPTOR provides minimal improvement over standard RAG for direct factual retrieval (“what is the specific value of X?”) where a leaf chunk contains the answer directly. For systems where most queries are specific and factual, the indexing cost of RAPTOR is not worth the complexity. For systems where users regularly ask broad thematic questions or need synthesis across many documents, RAPTOR is one of the highest-impact advanced techniques available.
Figure 1 — RAPTOR, HyDE, and RAG Fusion: When Each Technique Helps
HyDE: Hypothetical Document Embeddings in Depth
HyDE (Hypothetical Document Embeddings, introduced by Gao et al. 2022) inverts the standard retrieval logic. Standard RAG embeds the query and searches for similar document chunks. HyDE generates a hypothetical answer to the query using an LLM, embeds that hypothetical answer, and uses that vector for retrieval. The hypothetical answer is typically wrong in its specifics (the LLM is generating without access to your documents) but correct in style, vocabulary, and topic — which is exactly what you need for retrieval. A hypothetical answer shares the vocabulary and phrasing of real answer-containing chunks far better than the original question does.
Consider the vocabulary mismatch: a user asks “What is the deadline for expense submissions?” A document contains “Expense reports must be submitted within 30 days of the purchase date.” The question and the answer use completely different words. HyDE generates “Expense reports are due within a certain number of days after purchase” — hypothetical, probably wrong in the specific number, but using the vocabulary of the target document rather than the question. This hypothetical-answer vector retrieves the correct document chunk far more reliably than the original question vector.
import ollama
def hyde_retrieve(query: str, collection, n: int = 5,
llm_model: str = "llama3.1") -> list[str]:
"""HyDE retrieval: generate hypothetical answer, retrieve with its embedding."""
# Step 1: generate hypothetical answer
response = ollama.chat(
model=llm_model,
messages=[{"role": "user", "content":
f"Write a brief, direct answer to the following question as if you had "
f"access to a relevant document. Be specific but concise (2-3 sentences).\n\n"
f"Question: {query}\n\nHypothetical answer:"
}],
options={"temperature": 0.4, "num_predict": 150}
)
hypothetical = response["message"]["content"].strip()
# Step 2: embed hypothetical answer, not the original query
hypo_embedding = ollama.embeddings(
model="nomic-embed-text", prompt=hypothetical
)["embedding"]
# Step 3: retrieve using hypothetical embedding
results = collection.query(
query_embeddings=[hypo_embedding],
n_results=n
)
return results["documents"][0]
# Usage
retrieved = hyde_retrieve("What is the expense submission deadline?", collection)
HyDE works best when queries are short and stylistically different from the documents (users ask in natural question form, documents are written in statement form), when the document vocabulary is technical or domain-specific (the LLM’s hypothetical will use domain terms even without knowing the specific answer), and when retrieval quality with standard embedding is noticeably poor. HyDE can hurt quality if the LLM generates a confidently wrong hypothetical that pulls retrieval in the wrong direction. Test it on your specific query distribution before adopting it broadly.
RAG Fusion: Combining Multiple Retrieval Signals
RAG Fusion (introduced by Adrian Raudaschl in 2023, building on the RRF algorithm) addresses the problem that any single retrieval strategy has blind spots. RAG Fusion generates multiple query variants, retrieves for each independently, and combines the rankings using Reciprocal Rank Fusion to produce a final result list that is more robust than any single retrieval.
The algorithm: for a user query, generate 3-5 semantically varied reformulations using an LLM (different phrasing, different emphasis, synonyms). Run vector retrieval for each query variant. Apply Reciprocal Rank Fusion: for each unique retrieved chunk, sum 1/(k + rank) across all query variants where k=60 is a smoothing constant. Sort chunks by their total RRF score. Return the top-ranked chunks to the LLM. Chunks that appear in multiple query variants’ results get higher scores, surfacing content that is robustly relevant across different ways of expressing the same information need.
def generate_query_variants(query: str, n: int = 4) -> list[str]:
response = ollama.chat(
model="llama3.1",
messages=[{"role": "user", "content":
f"Generate {n} different ways to phrase this search query, "
f"one per line, no numbering or bullets:\n\n{query}"
}],
options={"temperature": 0.7, "num_predict": 200}
)
variants = [v.strip() for v in response["message"]["content"].split("\n") if v.strip()]
return [query] + variants[:n] # include original
def rag_fusion_retrieve(query: str, collection, n_final: int = 5,
n_per_query: int = 10, k: int = 60) -> list[str]:
"""RAG Fusion with Reciprocal Rank Fusion."""
variants = generate_query_variants(query)
rrf_scores: dict[str, float] = {}
chunk_texts: dict[str, str] = {}
for variant in variants:
v_embed = ollama.embeddings(
model="nomic-embed-text", prompt=variant
)["embedding"]
results = collection.query(
query_embeddings=[v_embed],
n_results=n_per_query
)
for rank, chunk in enumerate(results["documents"][0]):
rrf_scores[chunk] = rrf_scores.get(chunk, 0) + 1.0 / (k + rank + 1)
chunk_texts[chunk] = chunk
# Sort by RRF score and return top n_final
ranked = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)
return [chunk for chunk, _ in ranked[:n_final]]Combining Techniques: A Production Stack
These three techniques — RAPTOR, HyDE, and RAG Fusion — are not mutually exclusive. A production RAG system might use RAPTOR for the index (leaf chunks plus summary nodes), RAG Fusion at query time (multiple query variants), and reranking to select the final context (the previous guide covers reranking). Each addresses a different failure mode: RAPTOR for synthesis gaps, RAG Fusion for vocabulary and phrasing gaps, reranking for relevance precision. Combining them produces retrieval quality significantly better than any individual technique, at the cost of higher latency (each added query-time technique adds 1-3 seconds) and higher complexity. For most RAG systems, reranking alone produces the most gain for the least effort. Add RAPTOR if users frequently ask synthesis questions. Add RAG Fusion if queries are diverse and phrasing-dependent. Rarely do you need all three simultaneously — the marginal gain from the third technique is usually smaller than the added latency and code complexity cost.
Measuring the Impact of Each Technique
Before adding any of these techniques to a production system, measure the quality improvement on a representative test set. The measurement process: take 20-50 real user queries with known correct answers, run retrieval under your current setup and record whether the correct answer appears in the top-5 retrieved chunks (recall at 5), then apply the technique and re-run the same evaluation. A technique that increases recall from 70% to 85% is worth adopting. A technique that increases recall from 85% to 87% may not justify the latency and complexity cost. RAPTOR and RAG Fusion both add 1-3 seconds of latency per query — users notice latency increases of this magnitude in interactive applications. Batch processing pipelines are more tolerant. The right optimisation target depends on your use case: for interactive chat, minimise latency while maintaining acceptable quality; for background processing pipelines, maximise quality with less concern for latency.
Quick Implementation Guide: Adding Each Technique
For developers who have a working standard RAG pipeline and want to experiment with these techniques, here is the minimum viable implementation path for each. For RAPTOR: keep your existing leaf chunk index, add a clustering and summarisation step that generates summary nodes and inserts them into the same collection with a metadata tag indicating level. One afternoon of coding, one indexing run. For HyDE: replace the embed(query) call in your retrieve function with the two-step generate-then-embed approach. Fifteen lines of code, no index changes. For RAG Fusion: replace the single collection.query() call with the multi-query loop and RRF aggregation. Twenty lines of code, no index changes. Both HyDE and RAG Fusion are retrieval-time changes only — they require no re-indexing and can be A/B tested immediately by switching between the standard and enhanced retrieve function in your evaluation. RAPTOR requires a re-indexing run but the existing index can be extended rather than rebuilt. The low implementation cost of HyDE and RAG Fusion makes them easy first experiments when standard retrieval is underperforming.