mljourney

A complete guide to LM Studio for local LLMs: installing on Mac, Windows, and Linux, browsing and downloading models with RAM requirements shown upfront, loading models and adjusting generation parameters in the chat UI, starting the built-in OpenAI-compatible server on port 1234, connecting with the OpenAI Python SDK, tuning GPU layers and context length for your hardware, and a direct comparison of LM Studio versus Ollama to help you decide which fits your workflow.

A practical comparison of prefix tuning, prompt tuning, and P-Tuning v2: how each method works, where soft tokens are inserted, parameter counts, scaling behavior, and when to choose each over LoRA for multi-task serving from a single frozen model.

A practical guide to Ollama’s OpenAI-compatible API: using the OpenAI Python SDK pointed at localhost, streaming completions, generating embeddings with nomic-embed-text, switching existing OpenAI code to Ollama with two line changes, integrating with LangChain and LlamaIndex, using environment variables to toggle between local and cloud, and a clear summary of what the compatibility layer does and does not support.

A complete guide to fine-tuning Llama 3 using PyTorch FSDP across multiple GPUs: wrapping strategy with transformer_auto_wrap_policy, sharding strategies (FULL_SHARD vs HYBRID_SHARD), gradient checkpointing integration, bfloat16 training loop, full state dict checkpointing, and memory budget planning for 8B and 70B models.

A complete setup guide for Open WebUI with Ollama: installing via Docker with a single run command, pip installation without Docker, connecting to Ollama and troubleshooting disconnection issues, switching and pulling models from the UI, setting system prompts and custom personas, uploading documents for local RAG, accessing Open WebUI from other devices on your network, keeping conversations across updates, and the most useful settings to configure for a single-user local setup.

A practical guide to writing GPU kernels with OpenAI Triton: the tile-based programming model, a minimal working kernel, fused softmax, autotuning block sizes, 2D matrix kernels, autograd integration, debugging with the interpreter, and performance profiling against the memory roofline.

A practical guide to the best coding LLMs for local use in 2026: Qwen2.5-Coder 7B, 14B and 32B as the overall best across VRAM tiers, DeepSeek-Coder-V2 as a fast MoE option, Codestral 22B for fill-in-the-middle completions, hardware requirements at each tier from 8GB to 24GB VRAM, setting up Continue in VS Code with a local Ollama model, recommended Modelfiles with coding-optimised parameters, and how to choose the right model for your hardware and workflow.

Ground truth labels are unavailable at production scale for most LLM applications. A practical guide to reference-free evaluation: LLM-as-judge, G-Eval, consistency-based methods, behavioral testing, and how to layer them into a reliable evaluation stack.

LLM inference latency comes from two distinct phases with different bottlenecks. A practical guide to KV cache management, continuous batching, quantization, speculative decoding, and prefix caching — and how to combine them for your specific latency vs throughput target.

Prompt engineering alone is not enough for reliable JSON output. A practical guide to constrained decoding with Outlines and vLLM guided decoding, Pydantic + instructor for OpenAI, semantic validation with retry logic, and when to fine-tune a smaller model for your schema.