A complete guide to fine-tuning Llama 3 using PyTorch FSDP across multiple GPUs: wrapping strategy with transformer_auto_wrap_policy, sharding strategies (FULL_SHARD vs HYBRID_SHARD), gradient checkpointing integration, bfloat16 training loop, full state dict checkpointing, and memory budget planning for 8B and 70B models.
A complete setup guide for Open WebUI with Ollama: installing via Docker with a single run command, pip installation without Docker, connecting to Ollama and troubleshooting disconnection issues, switching and pulling models from the UI, setting system prompts and custom personas, uploading documents for local RAG, accessing Open WebUI from other devices on your network, keeping conversations across updates, and the most useful settings to configure for a single-user local setup.
A practical guide to writing GPU kernels with OpenAI Triton: the tile-based programming model, a minimal working kernel, fused softmax, autotuning block sizes, 2D matrix kernels, autograd integration, debugging with the interpreter, and performance profiling against the memory roofline.
A practical guide to the best coding LLMs for local use in 2026: Qwen2.5-Coder 7B, 14B and 32B as the overall best across VRAM tiers, DeepSeek-Coder-V2 as a fast MoE option, Codestral 22B for fill-in-the-middle completions, hardware requirements at each tier from 8GB to 24GB VRAM, setting up Continue in VS Code with a local Ollama model, recommended Modelfiles with coding-optimised parameters, and how to choose the right model for your hardware and workflow.
Ground truth labels are unavailable at production scale for most LLM applications. A practical guide to reference-free evaluation: LLM-as-judge, G-Eval, consistency-based methods, behavioral testing, and how to layer them into a reliable evaluation stack.
LLM inference latency comes from two distinct phases with different bottlenecks. A practical guide to KV cache management, continuous batching, quantization, speculative decoding, and prefix caching — and how to combine them for your specific latency vs throughput target.
Prompt engineering alone is not enough for reliable JSON output. A practical guide to constrained decoding with Outlines and vLLM guided decoding, Pydantic + instructor for OpenAI, semantic validation with retry logic, and when to fine-tune a smaller model for your schema.
A practical guide to writing custom CUDA kernels for PyTorch: project setup, the C++/CUDA extension mechanism, JIT compilation for fast iteration, autograd integration with gradcheck, thread indexing, memory coalescing, dtype dispatch, and debugging with compute-sanitizer.
A practical guide to Ollama Modelfiles: creating custom named models with persistent system prompts, setting temperature, context window, stop sequences and other inference parameters, four ready-to-use Modelfile templates for code review, JSON output, document summarisation, and low-RAM setups, using custom models through the Ollama REST API, seeding few-shot examples with MESSAGE, exporting and sharing Modelfiles with teammates, and the most common gotchas around context window memory, stop tokens, and reproducibility.
Most RAG failures are retrieval failures. A practical guide to the techniques that actually move the needle: hybrid BM25 + dense retrieval, reciprocal rank fusion, cross-encoder reranking, HyDE query transformation, and chunking strategies — with retrieval evaluation built in.
