Generative AI

A complete migration guide from OpenAI to Ollama: the two-line Python and JavaScript SDK change, streaming that works identically, batch embeddings with nomic-embed-text, an environment variable pattern for switching between cloud and local without code changes, LangChain migration via both the OpenAI-compatible endpoint and the native ChatOllama integration, LlamaIndex migration with OpenAILike, and a clear breakdown of what the compatibility layer supports versus where gaps exist including function calling, vision, and logprobs.

A practical guide to reward model training for ML engineers: preference data structure and quality requirements, Bradley-Terry pairwise loss and training objective, LoRA-based reward model fine-tuning, pairwise accuracy as a training metric, reward hacking failure modes and mitigations, best-of-N sampling as a simpler alternative to PPO, and when to use rejection sampling fine-tuning versus full RLHF.

A practical guide to using local LLMs for research paper workflows: fetching papers from arXiv with the Python library, extracting full text from PDFs with PyMuPDF, structured seven-point paper summarisation prompts, quick abstract-only relevance triage for reading list management, batch processing a list of arXiv IDs with Markdown output, an interactive Q&A session tool for deep reading, choosing between Llama 3.2 and Mistral Nemo for different paper lengths, and the privacy advantages of local processing for embargoed or proprietary research.

A technical deep-dive into diffusion models for ML engineers: the forward noising process and variance schedule, the reparameterisation trick for efficient training, ε-prediction U-Net training loop, DDIM deterministic sampling and fewer inference steps, latent diffusion and how Stable Diffusion works, and the score matching theory connecting it all.

A clear explanation of GGUF quantization formats in Ollama: what quantization does to model weights and memory, the full spectrum from Q2_K to F16 with sizes and VRAM requirements for a 7B model, why K-quantization is better than older Q4_0, a reference table of quality vs size trade-offs, when to use Q4_K_M vs Q5_K_M vs Q8_0, how to pull specific quantization tags, what Ollama pulls by default, and why lower quantization can actually be faster on memory-bandwidth-constrained consumer GPUs.

A practical guide to cross-encoder reranking for ML engineers building RAG systems: why bi-encoder retrieval misses relevant chunks, how cross-encoders score query-document pairs jointly, reranking with sentence-transformers ms-marco and BAAI/bge-reranker models, integrating via LangChain ContextualCompressionRetriever, latency and batching optimisation, and how to choose between open-source and hosted reranker options.

A complete guide to Open WebUI beyond the basics: multi-user management with admin and pending roles, configuring system prompts and custom models per use case, the document RAG library for team knowledge bases, web search integration with SearXNG or Bing, conversation branching and message editing, Arena mode for side-by-side model comparison, the Functions and Pipelines extensibility system, the OpenAI-compatible API with generated keys, and key admin settings to configure for team deployments.

A practical guide to RAG chunking strategies for ML engineers: recursive fixed-size chunking with token-aware overlap, semantic chunking via sentence-level similarity breakpoints, hierarchical parent-child chunking for precision-plus-context retrieval, document-aware splitting for structured corpora, and how to choose chunk size empirically using RAGAS context recall.

A practical guide to summarising meeting notes and transcripts locally with Ollama: a structured summarisation prompt with sections for overview, discussion points, decisions, action items, and open questions, transcribing recorded meetings with Whisper then summarising, extracting action items as structured JSON, generating follow-up emails from the summary, a complete command-line tool with argparse, choosing between Llama 3.2, Mistral Nemo and Qwen2.5, handling variable note formats, and the privacy case for keeping sensitive meeting content off cloud services.

A practical deep-dive into Mamba and state space models for ML engineers: the SSM recurrence and linear-time scaling, Mamba selective state spaces with input-dependent parameters, parallel scan CUDA kernels, Mamba vs transformer performance tradeoffs on recall and throughput, Mamba 2 and SSM variants including RWKV and Griffin, and when to reach for Mamba over a transformer in production.