Running a coding LLM locally means faster completions, no API costs, and full privacy for proprietary code. But not all coding models are worth running locally — some are too large for most hardware, others are optimised for benchmarks but weak on real tasks. This guide covers the best coding LLMs available to run locally in 2026, what hardware each requires, and how to get them running with Ollama in minutes.
What Makes a Good Local Coding LLM?
Three things matter most for a local coding assistant: code quality at small parameter counts (you want the best output possible within your RAM budget), inference speed (a model that takes 30 seconds per response breaks your flow), and context length (code tasks often require loading large files or multiple files simultaneously). The good news is that coding-specialised models have improved dramatically — a 7B coding model in 2026 produces output that matches or exceeds what a 13B general model produced two years ago for most programming tasks.
Qwen2.5-Coder: Best Overall
Qwen2.5-Coder from Alibaba is the strongest coding model family available for local use in 2026. The 7B variant fits comfortably on 8GB VRAM and produces code quality that rivals models twice its size. The 14B variant is excellent for 16GB VRAM setups. Both support a 128K token context window, which is critical for real codebases.
# Pull and run Qwen2.5-Coder 7B (fits on 8GB VRAM)
ollama pull qwen2.5-coder:7b
ollama run qwen2.5-coder:7b
# 14B for better quality on 16GB VRAM
ollama pull qwen2.5-coder:14b
# 32B for high-end setups (24GB+ VRAM)
ollama pull qwen2.5-coder:32bQwen2.5-Coder performs particularly well on Python, JavaScript, TypeScript, Java, C++, and Go. It handles code explanation, debugging, refactoring, and test generation reliably. The instruction-following is strong enough that it respects formatting constraints (“only return the function, no explanation”) more consistently than most alternatives.
DeepSeek-Coder-V2: Best for Complex Reasoning
DeepSeek-Coder-V2 is a Mixture-of-Experts architecture, which means it activates only a fraction of its parameters per token — making it faster and more memory-efficient than a dense model of equivalent capability. The 16B MoE variant runs on hardware that would normally require a 7B dense model.
# DeepSeek-Coder-V2 Lite (16B MoE, runs like a 2.4B dense model on RAM)
ollama pull deepseek-coder-v2:16b
ollama run deepseek-coder-v2:16b
# Full DeepSeek-Coder-V2 (236B MoE — needs multi-GPU or very high RAM)
ollama pull deepseek-coder-v2DeepSeek-Coder-V2 Lite is the sweet spot for most setups — it handles algorithmic problems, data structure implementation, and multi-file refactoring tasks better than same-size dense models. Its weakness is that very long context tasks occasionally lose coherence near the end of the context window.
Codestral: Best for Speed
Mistral’s Codestral 22B is one of the fastest coding models at its quality level. On a modern GPU it generates tokens noticeably faster than Qwen2.5-Coder at comparable quality. If inference latency is your primary concern — for example, you want completions in under a second — Codestral is worth benchmarking on your hardware.
ollama pull codestral:22b
ollama run codestral:22bCodestral is particularly strong on fill-in-the-middle (FIM) tasks — completing code given both the prefix and suffix context. This is the mode that code editors use for inline completions, making Codestral a good choice if you are setting up a local coding assistant in VS Code or Neovim via Continue or similar plugins.
Llama 3.1 / 3.2: Best General-Purpose with Code Capability
If you want a single model that handles both code and general tasks well rather than a pure coding specialist, Llama 3.2 8B Instruct is a strong choice. It is not as sharp as Qwen2.5-Coder on pure coding benchmarks, but it handles mixed tasks — explaining code, writing documentation, answering questions about code — more naturally than specialist models that are heavily tuned toward code generation.
ollama pull llama3.2:8b
ollama run llama3.2:8bHardware Requirements at a Glance
Here is what you realistically need for each tier:
8GB VRAM (RTX 3070/4060 Ti, M1/M2 Pro): Qwen2.5-Coder 7B Q4_K_M, DeepSeek-Coder-V2 16B (MoE), Llama 3.2 8B. These cover 90% of everyday coding tasks — single-file edits, function generation, debugging, test writing.
16GB VRAM (RTX 3080/4080, M2 Max/M3 Pro): Qwen2.5-Coder 14B, Codestral 22B Q4_K_M. Noticeably better at multi-file refactoring, complex algorithm implementation, and longer context tasks.
24GB VRAM (RTX 3090/4090, M2 Ultra): Qwen2.5-Coder 32B Q4_K_M. This is the sweet spot for serious local coding work — quality approaches GPT-4 class on many benchmarks at this size.
CPU-only or low VRAM: Qwen2.5-Coder 3B or Llama 3.2 3B. Usable for simple tasks on machines without a discrete GPU. Expect 3–8 tokens per second on a modern CPU, which is slow but functional for non-interactive use cases.
Setting Up a Coding Assistant in VS Code with Continue
Continue is a free, open-source VS Code and JetBrains plugin that integrates local Ollama models as a coding assistant — inline completions, chat sidebar, and code editing commands. Setting it up takes about five minutes.
# 1. Install Continue from VS Code marketplace
# Search: "Continue - Codestral, Claude, and more"
# 2. Make sure Ollama is running with your chosen model
ollama serve
ollama pull qwen2.5-coder:7bThen open Continue’s config file (~/.continue/config.json) and add your local model:
{
"models": [
{
"title": "Qwen2.5-Coder 7B",
"provider": "ollama",
"model": "qwen2.5-coder:7b",
"apiBase": "http://localhost:11434"
}
],
"tabAutocompleteModel": {
"title": "Qwen2.5-Coder 7B",
"provider": "ollama",
"model": "qwen2.5-coder:7b"
}
}After saving the config, the Continue sidebar activates in VS Code. You can chat with the model about your code, highlight a function and ask it to refactor or explain it, or use tab completion for inline suggestions. For completions specifically, Codestral 22B tends to give better single-line completions while Qwen2.5-Coder 7B is faster — try both and see which fits your workflow.
Recommended Modelfiles for Coding
A Modelfile lets you lock in the right system prompt and parameters for coding use. Here are two production-ready configs:
# Save as Modelfile.coding-assistant
FROM qwen2.5-coder:7b
SYSTEM """You are a senior software engineer. When asked to write code:
- Write clean, production-ready code only
- No explanations unless explicitly asked
- Use meaningful variable names
- Handle edge cases
- Follow the language's idiomatic style"""
PARAMETER temperature 0.15
PARAMETER num_ctx 32768
PARAMETER repeat_penalty 1.05ollama create coding-assistant -f Modelfile.coding-assistant
ollama run coding-assistantWhich Model Should You Start With?
The straightforward answer: start with Qwen2.5-Coder 7B if you have 8GB+ VRAM, or Qwen2.5-Coder 14B if you have 16GB+. It consistently tops local coding benchmarks at its size tier, has strong instruction following, and the Ollama pull is fast. If you find it too slow for interactive use, try DeepSeek-Coder-V2 16B — the MoE architecture gives you similar quality at higher token throughput. Add Codestral 22B to your toolkit specifically if you use VS Code completions heavily, since its fill-in-the-middle performance is best-in-class at a manageable size. Resist the temptation to immediately pull the largest model that fits in your RAM — the 7B and 14B models handle the vast majority of real coding tasks well, and the speed difference matters more than the marginal quality improvement when you are waiting for responses during active development.
Benchmarking Local Coding Models on Your Own Tasks
Public benchmarks like HumanEval and MBPP measure code generation accuracy on isolated function-writing tasks. They are useful proxies but do not fully predict how a model will perform on your actual work — which might involve a specific language, framework conventions, existing codebase context, or task types (debugging, refactoring, reviewing) that differ from benchmark tasks. The most reliable way to pick between two candidate models is to run them both on ten representative tasks from your own workflow and compare the outputs side by side.
A simple way to do this with Ollama is to write a short evaluation script that sends the same prompts to both models and writes the outputs to a file for comparison. This takes about fifteen minutes to set up and gives you a personalised benchmark that matters far more than HumanEval scores for your specific use case.
import requests
import json
MODELS = ['qwen2.5-coder:7b', 'deepseek-coder-v2:16b']
TASKS = [
'Write a Python function that parses a CSV file and returns a list of dicts',
'Refactor this function to handle the case where the input list is empty: def first(items): return items[0]',
'Write a SQL query to find the top 5 customers by total order value',
'Write a pytest test for a function that adds two numbers',
'Explain what this does and suggest improvements: for i in range(len(lst)): print(lst[i])',
]
def query(model, prompt):
r = requests.post('http://localhost:11434/api/chat',
json={'model': model, 'messages': [{'role':'user','content':prompt}], 'stream': False})
return r.json()['message']['content']
results = {}
for task in TASKS:
results[task] = {}
for model in MODELS:
results[task][model] = query(model, task)
print(f'Done: {model[:20]} on task {TASKS.index(task)+1}/{len(TASKS)}')
with open('model_comparison.json', 'w') as f:
json.dump(results, f, indent=2)
print('Results saved to model_comparison.json')Mac vs Windows vs Linux: Platform Considerations
Ollama’s performance characteristics differ meaningfully across platforms. On Apple Silicon (M1/M2/M3/M4), Ollama uses the Metal GPU backend and the unified memory architecture is a significant advantage — an M2 Max with 32GB unified memory can run a 14B model in Q8_0 quantisation entirely in GPU memory, while a Windows machine with a 16GB discrete GPU can only fit the same model in lower quantisation or must use CPU offloading. If you are choosing hardware specifically for local coding LLMs, Apple Silicon gives the best performance-per-dollar at the 8–32GB memory tier.
On Windows with an NVIDIA GPU, CUDA acceleration works out of the box with Ollama. Make sure you have the latest NVIDIA drivers installed — Ollama bundles its own CUDA libraries so you do not need to install the full CUDA toolkit separately. On Linux with NVIDIA hardware, performance is typically 5–15% faster than Windows for the same model and hardware, primarily due to lower driver overhead. AMD GPU support (ROCm) is available on Linux but requires more manual setup and has occasional compatibility issues with specific model quantisations — check the Ollama GitHub issues page for your specific GPU model before committing to an AMD setup for coding work.
Keeping Models Updated
Coding model weights improve frequently. Qwen2.5-Coder, DeepSeek-Coder, and Codestral all release updated versions regularly, and the improvements between minor versions are often significant for specific task types. Ollama makes updating straightforward — pulling a model that already exists locally re-downloads only the changed layers rather than the full weights, so updates are faster than the initial pull. A simple habit is to run ollama pull qwen2.5-coder:7b once a month to ensure you are on the latest version. The ollama list command shows the current digest for each model, and the Ollama model library page at ollama.com shows the latest available digest for comparison.
Local vs API: When Does Local Actually Win?
Running a coding LLM locally makes sense in specific circumstances. The clearest case is proprietary code you cannot send to an external API — internal tooling, unreleased products, code under strict NDA. The second case is high-frequency, low-latency use: if you want inline completions in your editor firing every few keystrokes, the round-trip latency to a remote API adds up, and a local model that is slower per token can still feel faster in practice because there is no network overhead. The third case is cost at scale: if you are generating large volumes of code — test suites, boilerplate, documentation — local inference has zero marginal cost once the hardware is paid for, while API costs scale linearly with usage.
Local does not win when your primary concern is the absolute best output quality regardless of cost — GPT-4o and Claude Sonnet still outperform 7B and 14B local models on complex multi-step coding problems, particularly those requiring deep reasoning across many files simultaneously. The gap narrows significantly at 32B and above, but most people with hardware capable of running a 32B model well are already in a professional context where API access is not the bottleneck. For most individual developers, the practical answer is: use a local 7B or 14B model for the majority of repetitive coding tasks, and reach for a cloud API for the genuinely hard problems.