mljourney

A complete guide to Msty, the local LLM desktop app built for multi-model comparison: connecting to existing Ollama models, running the same prompt against multiple models simultaneously, attaching document Knowledge collections to conversations, conversation branching for exploring different approaches, the prompts library for reusable system prompts, integrating cloud providers like OpenAI and Anthropic alongside local models, and how Msty compares to Jan, LM Studio, and Open WebUI.

A practical deep-dive into Mixture of Experts architecture: how token routing works, load balancing with auxiliary loss, why MoE models are harder to serve than dense models despite lower per-token compute, expert parallelism tradeoffs, and when to choose MoE vs dense models for inference.

A complete setup guide for the Continue VS Code extension with Ollama: installing Continue and configuring config.json with chat and autocomplete models, enabling tab completions with fill-in-the-middle using Qwen2.5-Coder, indexing your codebase with nomic-embed-text embeddings for @codebase search, using built-in slash commands for editing, testing and commenting code, creating custom slash commands, switching between multiple local and cloud models, JetBrains setup, and troubleshooting connection issues.

A practical guide to instrumenting LLM applications with OpenTelemetry: setting up the SDK and OTLP exporter, creating spans around LLM calls with GenAI semantic conventions, tracing multi-step RAG pipelines with nested spans, what attributes to record, backend options from Grafana Tempo to Honeycomb and Langfuse, and when auto-instrumentation is enough vs when to add manual spans.

A practical guide to the best open-source LLMs in 2026 organised by use case and hardware tier: Llama 3.3 70B for best overall quality, Llama 3.2 8B for most users, Qwen2.5-Coder for coding, Phi-4 Mini and Qwen2.5 3B for small/CPU setups, Gemma 3 27B for long context, DeepSeek-R1 for reasoning, Qwen2.5 for multilingual use, and a quick reference table matching models to hardware from 4GB to 48GB RAM.

A technical deep dive into Mixture of Experts (MoE) architecture: how routers select experts per token, why expert collapse happens and how load balancing loss fixes it, memory vs compute tradeoffs compared to dense models, inference challenges with batching and expert parallelism, and when to choose MoE over a dense model for your serving setup.

A complete guide to Jan, the open-source local LLM desktop app: installing with no dependencies, browsing and downloading models from the built-in hub, importing local GGUF files, using the chat interface with file attachments, enabling the OpenAI-compatible API on port 1337, hardware detection and GPU layer configuration, how Jan compares to Ollama and LM Studio, and the extension system for adding remote API support and document Q&A.

A deep technical explainer on continuous batching for LLM inference: why static batching wastes GPU compute on autoregressive generation, how iteration-level scheduling works, the prefill vs decode phase distinction, PagedAttention and KV cache memory management, throughput vs latency tradeoffs, and vLLM configuration parameters for tuning continuous batching in production.

A practical guide to AnythingLLM for local document chat: desktop app and Docker installation, connecting to Ollama or LM Studio as the LLM backend, creating workspaces with isolated document collections, uploading PDFs and URLs for RAG, querying documents with source citations, using agents with web search and code execution, setting up multi-user access with role-based permissions, and a direct comparison with Open WebUI to help you choose the right tool for your workflow.

A complete guide to deploying machine learning models on Kubernetes using KServe: cluster setup, packaging PyTorch and HuggingFace models, InferenceService manifests, autoscaling with Knative, canary deployments for safe rollouts, and when KServe is worth the operational overhead versus simpler alternatives.