Standard vector RAG retrieves semantically similar text chunks. GraphRAG instead builds a knowledge graph from your documents and traverses entity relationships to find relevant information. Multimodal RAG extends the pipeline to images, tables, and mixed-content documents. These are not replacements for vector RAG — they solve specific problems that vector RAG handles poorly.
The Limits of Vector RAG
Vector RAG works well for direct question-answer retrieval: “what is the expense limit for team meals?” finds the answer via embedding similarity. It struggles with three specific question types. Multi-hop questions require connecting information across entities: “what projects did employees who joined in 2023 work on?” requires knowing who joined in 2023, then finding their projects — two retrieval steps with a join between them. Vector RAG retrieves fragments without following relational chains. Thematic synthesis questions ask about patterns across many documents: “what are the common failure modes across all our incident reports?” — no single chunk contains the answer. Entity-centric questions ask about everything related to a specific entity: “tell me everything about Project Phoenix” — a vector search may miss mentions spread across sections using varied phrasing. GraphRAG addresses these failure modes by building explicit entity relationships that can be traversed logically rather than retrieved by similarity.
How GraphRAG Works
Microsoft’s GraphRAG (open-sourced in 2024) processes documents in two stages. The indexing stage uses an LLM to extract entities (people, organisations, projects, concepts, dates) and relationships (worked-on, reported-to, caused-by) from every chunk, builds a knowledge graph connecting all entities, and generates community summaries — LLM-written summaries of each cluster of related entities in the graph. The query stage has two modes: local search retrieves entity-specific information by navigating the graph outward from query entities; global search synthesises across community summaries to answer broad thematic questions.
The trade-offs are significant. GraphRAG indexing is expensive — it requires many LLM calls to extract entities and generate summaries, 10-100x more costly than vector RAG. For a collection of a few thousand pages, indexing may take hours. Using local Ollama inference instead of cloud APIs dramatically reduces cost. Query quality for multi-hop and thematic questions substantially exceeds vector RAG; for direct factual questions the quality is similar.
pip install graphrag
# Initialise project
python -m graphrag init --root ./my-graphrag-project
# Place documents in ./my-graphrag-project/input/
# Configure settings.yaml (LLM endpoint, embedding model)
# Index (builds the knowledge graph)
python -m graphrag index --root ./my-graphrag-project
# Query - local (entity-specific)
python -m graphrag query \
--root ./my-graphrag-project \
--method local \
--query "What projects did the engineering team work on in Q3?"
# Query - global (thematic synthesis)
python -m graphrag query \
--root ./my-graphrag-project \
--method global \
--query "What are the common themes across all project retrospectives?"GraphRAG vs Vector RAG: When to Use Each
The choice should be driven by the question types your users actually ask. If most questions are direct factual retrieval, vector RAG with good chunking and reranking delivers excellent results at a fraction of the cost and complexity. If users frequently ask multi-hop questions, want synthesis across many documents, or need entity-centric queries, GraphRAG produces substantially better answers. Many production systems run both in parallel and route queries based on type — direct questions to vector RAG, synthesis and relational questions to GraphRAG.
Figure 1 — GraphRAG vs Vector RAG: Strengths by Query Type
Multimodal RAG: Images and Tables
Standard RAG processes text. Multimodal RAG extends retrieval and generation to images, charts, diagrams, and structured tables embedded in documents. There are two approaches, each with different trade-offs.
Text-based multimodal RAG converts non-text content into text before embedding: use a vision LLM to generate alt-text descriptions of images, and parse tables into markdown or CSV format. Both are then embedded as text alongside regular document chunks. This approach is simple to implement and works with any embedding and retrieval model. The limitation: the text description of an image loses visual information that may be needed to answer visual questions accurately.
Multimodal embedding RAG embeds images directly using a multimodal embedding model that produces vectors for both text and images in a shared embedding space. CLIP-family models and newer multimodal embeddings (ColPali, BGE-M3) allow retrieving images by text query and text by image query from the same index. This approach preserves visual information but requires a multimodal embedding model and a vector database that supports multiple vector types per document.
pip install pdf2image Pillow
from pdf2image import convert_from_path
import ollama, base64, io
def extract_page_text_and_images(pdf_path: str) -> list[dict]:
"""Extract both text and visual descriptions from PDF pages."""
import pypdf
pages = []
reader = pypdf.PdfReader(pdf_path)
pdf_images = convert_from_path(pdf_path, dpi=150)
for i, (page, img) in enumerate(zip(reader.pages, pdf_images)):
text = page.extract_text() or ""
# Get visual description from vision LLM
buf = io.BytesIO()
img.save(buf, format="PNG")
b64 = base64.b64encode(buf.getvalue()).decode()
vision_response = ollama.chat(
model="llama3.2-vision:11b",
messages=[{"role": "user",
"content": "Describe the key information in this page, "
"including any charts, tables or diagrams.",
"images": [b64]}]
)
visual_desc = vision_response["message"]["content"]
pages.append({
"page": i + 1,
"text": text,
"visual_description": visual_desc,
"combined": f"{text}\n\n[Visual content: {visual_desc}]"
})
return pages
# Index combined text+visual descriptions
pages = extract_page_text_and_images("annual_report.pdf")
for page in pages:
chunks = chunk_text(page["combined"])
embeddings = [embed(c) for c in chunks]
collection.add(
ids=[f"report_p{page['page']}_{i}" for i in range(len(chunks))],
embeddings=embeddings,
documents=chunks,
metadatas=[{"source": "annual_report", "page": page["page"]} for _ in chunks]
)Table-Aware RAG
Tables in PDFs are one of the most common failure modes in standard RAG — PDF text extraction produces garbled table content that embeds poorly and retrieves unreliably. The best approaches for table-heavy documents: Table extraction tools like camelot or pdfplumber extract tables into structured DataFrames that can be serialised to markdown or CSV before indexing. Vision-based extraction uses a vision LLM to read and describe tables from page screenshots, converting visual table structure into text that embeds correctly. Structured data RAG extracts tables into a SQL database or pandas DataFrame and generates SQL queries rather than retrieving text chunks — precise for numerical and filtering queries but requires more upfront data modelling work.
When to Use Multimodal RAG
Multimodal RAG is worth the additional complexity when your documents contain significant non-text content that users need to query: financial reports with charts, technical manuals with diagrams, scientific papers with figures, or any document where the visual content carries meaning that text extraction misses. For mostly-text documents with occasional images, standard text extraction with a brief visual description pass is usually sufficient. For documents where the charts and tables are the primary information source — think quarterly earnings reports or engineering specifications — multimodal extraction is necessary for acceptable answer quality. The vision LLM description approach (converting visuals to text) is the most practical starting point: it works with any downstream vector database and retrieval pipeline, requires only adding a vision model to your existing setup, and produces good results for most document types without requiring specialised multimodal embedding infrastructure.
The Evolving RAG Landscape
RAG architectures are evolving rapidly as both LLM context windows and retrieval techniques improve. Long-context LLMs (1M+ token context in some frontier models) enable an alternative to retrieval: simply load the entire document corpus into context and let the model find relevant information natively. For small to medium document collections this is increasingly practical and eliminates the retrieval quality problems that advanced RAG techniques are designed to address. The trade-off is cost and latency at scale — processing millions of tokens per query is expensive and slow compared to retrieval-then-generate. The practical picture in 2026: vector RAG with reranking handles most production use cases well. GraphRAG solves specific multi-hop and synthesis problems. Long-context models are becoming viable for smaller corpora. Multimodal RAG extends coverage to visual documents. The right architecture depends on your specific documents, queries, and scale constraints — and that assessment is worth making carefully before investing in complexity.
Using GraphRAG with Local Ollama Models
GraphRAG’s default configuration targets cloud LLM APIs (OpenAI, Azure OpenAI). Configuring it to use Ollama instead dramatically reduces indexing cost and keeps the entire pipeline local. The key configuration in settings.yaml: set the LLM type to openai_chat with Ollama’s base URL, specify your Ollama model name, and set the embedding model to the Ollama embedding endpoint. Llama 3.1 8B handles entity extraction and summarisation adequately for most document types, though 70B produces higher quality entity extraction on complex documents with many named entities and nuanced relationships. The indexing pipeline is the same regardless of which LLM powers it — Ollama simply acts as a drop-in replacement for the OpenAI API endpoint in the configuration. Community guides for GraphRAG with Ollama are available on GitHub and the GraphRAG repository Issues page, as the configuration details change with GraphRAG releases. The local-first GraphRAG setup is particularly valuable for organisations with document confidentiality requirements where sending documents through cloud LLM APIs for entity extraction is not acceptable.
Agentic RAG: When Retrieval Is One Step in a Larger Pipeline
Advanced RAG pipelines increasingly blur into agentic workflows. Agentic RAG allows an LLM to decide when to retrieve, what to retrieve, and whether the retrieved information is sufficient before answering. Rather than a fixed retrieve-then-generate pipeline, an agentic system might: retrieve initial context, determine it is insufficient, retrieve again with a refined query, check whether the retrieved content actually answers the question, and either answer or retrieve again. This iterative, self-directed retrieval is more expensive per query but substantially more capable for complex questions requiring multiple retrieval steps. LangGraph and LlamaIndex’s AgentWorkflow provide frameworks for implementing agentic RAG pipelines. For most applications, fixed-pipeline RAG (even with advanced techniques like reranking and parent-child chunking) is sufficient and more predictable. Agentic RAG earns its complexity when queries are genuinely diverse, unpredictable, and require adaptive retrieval strategies that cannot be hardcoded into a fixed pipeline.