Mistral Small 3 is one of the most interesting models to land in the local LLM space in 2025. At 24 billion parameters, it sits in an underserved spot — larger than the 7B class where most consumer hardware plays, but far more accessible than 70B models that require server-grade RAM. What makes it notable … Read more
WSL2 (Windows Subsystem for Linux 2) gives you a full Linux kernel running inside Windows — and Ollama on WSL2 is a genuinely good setup. You get native Linux performance, GPU passthrough for NVIDIA cards, and the ability to run Ollama alongside your Linux development toolchain without leaving Windows. If you’re a developer who primarily … Read more
Phi-4 is Microsoft’s answer to a deceptively simple question: how capable can a small model get? At 14 billion parameters, it competes with — and often beats — models two to four times its size on reasoning and coding benchmarks. On a Mac with 16GB unified memory or a PC with a decent GPU, Phi-4 … Read more
Llama 3.3 70B is the largest model most people can realistically run on consumer hardware — and it’s genuinely impressive. Meta’s instruction-tuned 70B model sits in a category by itself among open-source models: strong enough to compete with frontier API models on many tasks, but running entirely on your machine. The catch is that you … Read more
Qwen 2.5 Coder is one of the best coding models you can run locally right now. Alibaba’s coding-specific fine-tune of Qwen 2.5 punches well above its weight class — the 7B version outperforms models twice its size on most coding benchmarks, and the 14B version is genuinely competitive with GPT-4o mini on code tasks. With … Read more
Keeping Ollama updated matters more than it might seem. New versions bring faster inference, support for newly released models, bug fixes for GPU detection issues, and sometimes significant performance improvements for specific hardware. The update process is quick on all platforms — usually under two minutes — and your downloaded models are never touched during … Read more
The M4 chip made running serious LLMs on a MacBook genuinely practical. Unified memory means the GPU and CPU share the same pool — no copying data back and forth between VRAM and system RAM — and the memory bandwidth on even the base M4 is fast enough to make 7B and 13B models feel … Read more
Ubuntu 24.04 is one of the best environments for running Ollama — the installation is clean, GPU support via CUDA works reliably, and you can run it headlessly on a server or as a persistent service without any drama. Whether you’re setting it up on a local workstation, a home server, or a cloud VM, … Read more
Running large language models locally on Windows used to be a painful experience — CUDA drivers, Python environments, mysterious errors. Ollama changed that. It wraps everything into a single installer and gives you a clean CLI and API to pull and run models in minutes. This guide walks you through installing Ollama on Windows 11, … Read more
Why Memory Is the Hard Problem in AI Agents LLMs are stateless by default — every request arrives without any memory of previous interactions unless the relevant history is explicitly included in the context. For simple chatbots, this is manageable: include the last N conversation turns and the user’s question feels contextually grounded. For AI … Read more
