A practical guide to instrumenting LLM applications with OpenTelemetry: setting up the SDK and OTLP exporter, creating spans around LLM calls with GenAI semantic conventions, tracing multi-step RAG pipelines with nested spans, what attributes to record, backend options from Grafana Tempo to Honeycomb and Langfuse, and when auto-instrumentation is enough vs when to add manual spans.
A practical guide to the best open-source LLMs in 2026 organised by use case and hardware tier: Llama 3.3 70B for best overall quality, Llama 3.2 8B for most users, Qwen2.5-Coder for coding, Phi-4 Mini and Qwen2.5 3B for small/CPU setups, Gemma 3 27B for long context, DeepSeek-R1 for reasoning, Qwen2.5 for multilingual use, and a quick reference table matching models to hardware from 4GB to 48GB RAM.
A technical deep dive into Mixture of Experts (MoE) architecture: how routers select experts per token, why expert collapse happens and how load balancing loss fixes it, memory vs compute tradeoffs compared to dense models, inference challenges with batching and expert parallelism, and when to choose MoE over a dense model for your serving setup.
A complete guide to Jan, the open-source local LLM desktop app: installing with no dependencies, browsing and downloading models from the built-in hub, importing local GGUF files, using the chat interface with file attachments, enabling the OpenAI-compatible API on port 1337, hardware detection and GPU layer configuration, how Jan compares to Ollama and LM Studio, and the extension system for adding remote API support and document Q&A.
A deep technical explainer on continuous batching for LLM inference: why static batching wastes GPU compute on autoregressive generation, how iteration-level scheduling works, the prefill vs decode phase distinction, PagedAttention and KV cache memory management, throughput vs latency tradeoffs, and vLLM configuration parameters for tuning continuous batching in production.
A practical guide to AnythingLLM for local document chat: desktop app and Docker installation, connecting to Ollama or LM Studio as the LLM backend, creating workspaces with isolated document collections, uploading PDFs and URLs for RAG, querying documents with source citations, using agents with web search and code execution, setting up multi-user access with role-based permissions, and a direct comparison with Open WebUI to help you choose the right tool for your workflow.
A complete guide to deploying machine learning models on Kubernetes using KServe: cluster setup, packaging PyTorch and HuggingFace models, InferenceService manifests, autoscaling with Knative, canary deployments for safe rollouts, and when KServe is worth the operational overhead versus simpler alternatives.
A complete guide to LM Studio for local LLMs: installing on Mac, Windows, and Linux, browsing and downloading models with RAM requirements shown upfront, loading models and adjusting generation parameters in the chat UI, starting the built-in OpenAI-compatible server on port 1234, connecting with the OpenAI Python SDK, tuning GPU layers and context length for your hardware, and a direct comparison of LM Studio versus Ollama to help you decide which fits your workflow.
A practical comparison of prefix tuning, prompt tuning, and P-Tuning v2: how each method works, where soft tokens are inserted, parameter counts, scaling behavior, and when to choose each over LoRA for multi-task serving from a single frozen model.
A practical guide to Ollama’s OpenAI-compatible API: using the OpenAI Python SDK pointed at localhost, streaming completions, generating embeddings with nomic-embed-text, switching existing OpenAI code to Ollama with two line changes, integrating with LangChain and LlamaIndex, using environment variables to toggle between local and cloud, and a clear summary of what the compatibility layer does and does not support.
