Generative AI

A practical guide to running vision models locally with Ollama: available models including LLaVA, moondream2, Qwen2.5-VL and Gemma 3, using vision models from the CLI with image flags, calling them via the Ollama Python library and the OpenAI-compatible API with base64-encoded images, batch processing a folder of images, and practical use cases including screenshot OCR, chart analysis, document data extraction, and inventory cataloguing.

A practical comparison of Pinecone, Weaviate, and Qdrant for production RAG systems: architecture, filtered search performance, hybrid BM25 + vector search, cost at scale, ecosystem integrations, and a decision framework for choosing between fully managed and self-hosted vector databases.

A complete guide to running Ollama in Docker: CPU-only and NVIDIA GPU passthrough with the NVIDIA Container Toolkit, AMD ROCm setup, pulling models into the running container, a full Docker Compose stack combining Ollama with Open WebUI, environment variables for parallel requests and model keep-alive, persisting models across redeployments with named volumes, and when to use Docker versus native Ollama installation.

A practical comparison of sparse retrieval (BM25), dense retrieval (embeddings), and hybrid search for RAG pipelines: how each method works, where each fails, reciprocal rank fusion, learned sparse models like SPLADE and ColBERT, reranking, and how to choose the right configuration for your corpus and query distribution.

A complete guide to Msty, the local LLM desktop app built for multi-model comparison: connecting to existing Ollama models, running the same prompt against multiple models simultaneously, attaching document Knowledge collections to conversations, conversation branching for exploring different approaches, the prompts library for reusable system prompts, integrating cloud providers like OpenAI and Anthropic alongside local models, and how Msty compares to Jan, LM Studio, and Open WebUI.

A practical deep-dive into Mixture of Experts architecture: how token routing works, load balancing with auxiliary loss, why MoE models are harder to serve than dense models despite lower per-token compute, expert parallelism tradeoffs, and when to choose MoE vs dense models for inference.

A complete setup guide for the Continue VS Code extension with Ollama: installing Continue and configuring config.json with chat and autocomplete models, enabling tab completions with fill-in-the-middle using Qwen2.5-Coder, indexing your codebase with nomic-embed-text embeddings for @codebase search, using built-in slash commands for editing, testing and commenting code, creating custom slash commands, switching between multiple local and cloud models, JetBrains setup, and troubleshooting connection issues.

A practical guide to instrumenting LLM applications with OpenTelemetry: setting up the SDK and OTLP exporter, creating spans around LLM calls with GenAI semantic conventions, tracing multi-step RAG pipelines with nested spans, what attributes to record, backend options from Grafana Tempo to Honeycomb and Langfuse, and when auto-instrumentation is enough vs when to add manual spans.

A practical guide to the best open-source LLMs in 2026 organised by use case and hardware tier: Llama 3.3 70B for best overall quality, Llama 3.2 8B for most users, Qwen2.5-Coder for coding, Phi-4 Mini and Qwen2.5 3B for small/CPU setups, Gemma 3 27B for long context, DeepSeek-R1 for reasoning, Qwen2.5 for multilingual use, and a quick reference table matching models to hardware from 4GB to 48GB RAM.

A technical deep dive into Mixture of Experts (MoE) architecture: how routers select experts per token, why expert collapse happens and how load balancing loss fixes it, memory vs compute tradeoffs compared to dense models, inference challenges with batching and expert parallelism, and when to choose MoE over a dense model for your serving setup.