Basic RAG — chunk, embed, retrieve top-K, generate — works well for straightforward document Q&A. But production RAG systems encounter harder queries that basic retrieval handles poorly: questions that span multiple documents, questions where the most semantically similar chunks aren’t the most relevant, questions requiring temporal or structural reasoning, and ambiguous queries where the user’s … Read more
LangChain wraps ChromaDB (and most other vector databases) behind a unified interface, making it easy to swap components and prototype quickly. But LangChain adds abstraction overhead, makes debugging harder, and changes frequently enough that tutorials become outdated quickly. This guide covers both approaches: the LangChain integration for teams already invested in the ecosystem, and the … Read more
RAG (Retrieval-Augmented Generation) lets you give a language model access to your own documents — PDFs, notes, wikis, code — without fine-tuning. The model answers questions using content retrieved from your document store rather than only its training data. This guide builds a complete local RAG pipeline: Ollama for LLM inference, ChromaDB for vector storage, … Read more
pgvector extends PostgreSQL with vector similarity search, letting you store embeddings alongside your regular relational data and query them with standard SQL. For teams already running PostgreSQL, it is the lowest-friction path to adding vector search to an existing system — no new database to operate, no data synchronisation between services, and familiar tooling. But … Read more
Choosing a vector database is one of the first architectural decisions in any RAG or semantic search project, and the options have proliferated significantly. ChromaDB, Qdrant, Weaviate, and Pinecone are the most commonly evaluated — each with a distinct positioning. This guide compares them on the dimensions that actually matter for most projects: ease of … Read more
Windows is a perfectly capable platform for local LLM inference, with access to the full NVIDIA CUDA ecosystem, a choice between native Windows and WSL2 Linux environments, and dedicated GUI tools that make getting started straightforward. This guide covers the complete Windows local AI setup: CUDA drivers, Ollama on Windows, WSL2 as an alternative, and … Read more
ExLlamaV2 is an inference engine specifically optimised for NVIDIA GPUs running GPTQ-quantized models. If you have an NVIDIA GPU and want the fastest possible local LLM inference, ExLlamaV2 deserves serious consideration alongside llama.cpp. This guide compares the two engines on what actually matters for practical use: speed, model compatibility, ease of use, and ecosystem integration. … Read more
Two of the most practical aspects of llama.cpp that trip up new users: understanding how context length works and how to get the best out of it, and setting up llama-server as a persistent OpenAI-compatible API endpoint. This guide covers both thoroughly — the theory, the commands, and the trade-offs. How Context Length Works in … Read more
GGUF quantization format names — Q4_K_M, Q5_K_S, Q8_0, IQ2_XS — are cryptic at first glance but follow a consistent pattern once you understand the system. Choosing the right quantization is one of the most impactful decisions in local LLM deployment: it determines memory usage, inference speed, and output quality. This guide decodes the naming system, … Read more
llama.cpp and Ollama are not alternatives to each other — Ollama wraps llama.cpp. But developers often face a choice between using Ollama’s API and using llama.cpp directly through its Python bindings. This guide covers the real differences in speed, flexibility, and developer experience, and when the llama-cpp-python library is the better choice over the Ollama … Read more
