How to Use Aider with a Local LLM

by

Running Aider with a local LLM means you get AI-assisted coding with no API costs, no data leaving your machine, and no dependence on third-party services. The tradeoff is quality — local models are not yet as capable as frontier cloud models for complex coding tasks. But for the right use cases, the combination of Aider’s workflow and a capable local model like Qwen2.5-Coder or DeepSeek-Coder is genuinely productive, especially when privacy or cost is a constraint.

Why Use a Local Model with Aider

There are three situations where a local model makes sense over a cloud API:

  • Cost at scale — if you run many Aider sessions daily, API costs add up. A local model has zero marginal cost after the hardware.
  • Privacy — codebases with proprietary algorithms, client data, or regulated content should not leave your environment. Local models solve this cleanly.
  • Offline work — traveling, restricted networks, or environments without reliable internet make local models the only option.

The quality gap is real but narrowing. As of 2026, a well-quantised Qwen2.5-Coder-32B running locally handles routine coding tasks — adding functions, writing tests, small refactors — competently. It struggles with complex multi-file reasoning that frontier models handle well. Know what you are trading before you switch.

Setting Up Ollama

Ollama is the simplest way to run local models and integrates directly with Aider. Install it from ollama.ai, then pull the best available coding model for your hardware:

# Best for 24GB VRAM — strong coding capability
ollama pull qwen2.5-coder:32b

# Good for 16GB VRAM — decent quality, fast
ollama pull qwen2.5-coder:14b

# Works on 8GB VRAM — limited but functional
ollama pull qwen2.5-coder:7b

Ollama starts a local API server on localhost:11434 that is compatible with OpenAI’s API format. Aider connects to it using the same protocol it uses for cloud models — no special configuration required beyond pointing it at the right endpoint.

Connecting Aider to Ollama

Start Aider with the --model flag pointing to your Ollama model:

aider --model ollama/qwen2.5-coder:32b

Or set it persistently in .aider.conf.yml in the project root:

model: ollama/qwen2.5-coder:32b

That is all the configuration needed. Aider detects Ollama’s OpenAI-compatible endpoint automatically when the model name starts with ollama/. Run a quick test by opening a project and asking Aider to explain a function — if it responds coherently, the connection is working.

Figure 1 — Local model options for Aider by hardware tier

Hardware Recommended model Quality RTX 4090 / M2 Max 36GBQwen2.5-Coder:32bStrong — handles most tasks RTX 3080 / M1 Pro 16GBQwen2.5-Coder:14bGood for routine tasks 8GB VRAM / CPU onlyQwen2.5-Coder:7bLimited — simple changes only Multi-GPU (2× 24GB)DeepSeek-Coder-V2:236bNear-frontier quality

Which Local Models Work Best

Not all local models are equal for Aider’s use case. The key requirement is that the model reliably produces code changes in the diff format Aider expects — some general-purpose models get confused by this format even when they can write good code in isolation.

The models that consistently work well with Aider:

  • Qwen2.5-Coder (7b, 14b, 32b) — the current best-performing local coding model family. The 32b version handles multi-file edits reliably. Strong on Python, TypeScript, and Go.
  • DeepSeek-Coder-V2 — excellent for complex reasoning tasks, especially algorithm implementation. Requires more VRAM but produces frontier-quality output on hard tasks.
  • Codestral (Mistral’s coding model) — fast and accurate for file-level edits. Available via Ollama or Mistral’s API with better licensing for commercial use than some alternatives.

Models to avoid with Aider: general-purpose models like Llama-3 and Mistral-7B often struggle to produce clean diffs. They can answer questions about code but frequently produce malformed edit blocks that Aider cannot apply. Stick to models explicitly trained for code generation.

Context Window Limitations

This is the most important practical constraint when running local models. Frontier cloud models have context windows of 100k–200k tokens. Most local models max out at 8k–32k tokens, which means:

  • You can add fewer files to the context before hitting the limit
  • Long files may need to be split or summarised before being added
  • Multi-file refactors that work cleanly with Claude may fail with a local model

The practical workaround is stricter context discipline. Add only the specific functions or classes relevant to the task rather than entire files. Use /read instead of /add for files you only need the AI to understand, not modify — read-only files consume less context. Split large tasks into smaller steps with fewer files per step. This discipline is good practice anyway, but with local models it becomes necessary rather than optional.

Performance Expectations

Local model inference is slower than cloud APIs. On an RTX 4090 with Qwen2.5-Coder:32b at 4-bit quantisation, expect roughly 15–25 tokens per second. A typical Aider response generating a modified function might take 10–30 seconds. That is noticeably slower than cloud APIs (which return in 2–5 seconds for similar tasks) but perfectly usable for offline or privacy-sensitive work.

On Apple Silicon, the unified memory architecture helps significantly. An M2 Max with 36GB unified memory runs Qwen2.5-Coder:32b at around 20 tokens per second — comparable to GPU inference. The M3 Ultra with 64GB or 128GB unified memory can run even larger models with good performance, making high-end Macs a genuinely viable platform for local AI coding.

Hybrid Workflow: Local for Exploration, Cloud for Delivery

One effective pattern is using local models during active development and exploration — where you are iterating quickly, the code is not yet ready for anyone else to see, and you want zero latency and zero cost for the tight loop — then switching to a cloud model for the final polishing and complex multi-file changes before submitting a PR.

Aider makes this easy. Switch models mid-session with:

/model claude-sonnet-4-6

The conversation history and added files are preserved — only the model changes. You can start a session locally, get 80% of the feature working, then switch to Claude Sonnet for the final cleanup and edge case handling. This hybrid approach keeps costs down while still getting frontier quality where it matters most.

Troubleshooting Common Issues

A few issues come up regularly when setting up Aider with local models:

  • Malformed edit blocks — the model produces code but Aider cannot apply it. Usually means the model is not following Aider’s diff format. Try adding “respond only in the Aider edit format” to your system prompt, or switch to a more capable model.
  • Context window errors — Aider reports that the context is too long. Drop files with /drop until the error clears, or split the task into smaller steps.
  • Slow responses — normal for large models. If response time is unacceptable, try a smaller quantisation (Q3 instead of Q4) or a smaller model variant.
  • Ollama not found — Aider cannot connect to Ollama. Verify Ollama is running with ollama list and that the model name in your Aider config matches exactly what Ollama shows.

Running Aider with a local model is one of the most practical setups for developers who want AI coding assistance with full data sovereignty. The quality ceiling is lower than frontier cloud models, but for a wide range of everyday coding tasks — writing tests, adding validation, small refactors, explaining code — a well-chosen local model running through Aider is a genuinely useful tool that costs nothing to run once the hardware is in place.

Comparing Local vs Cloud Quality in Practice

The quality gap between local models and cloud APIs shows up unevenly across task types. For some tasks the difference is barely noticeable; for others it is significant enough to change whether AI assistance is useful at all.

Where local models hold up well:

  • Writing unit tests for existing functions — the model sees the function signature and docstring and produces reasonable test cases
  • Adding docstrings and type annotations — pattern-matching heavy, and smaller models are good at patterns
  • Renaming variables and small refactors within a single file
  • Explaining what a block of code does
  • Generating boilerplate — CRUD endpoints, data models, configuration files

Where the gap is most visible:

  • Multi-file refactors that require understanding how components relate across the codebase
  • Bug fixes where the root cause involves subtle interactions between files
  • Implementing an algorithm from a description when the implementation is non-trivial
  • Anything requiring significant context — the 8k–32k context limit means the model cannot see enough of the codebase at once to reason across it effectively

A pragmatic approach: use the local model for the tasks where it works well and switch to cloud for the hard ones. The cost of a single complex Claude session is often trivial compared to the time saved, and knowing which tasks exceed local model capability lets you make that switch deliberately rather than after wasting time with poor-quality output.

Security and Privacy Benefits in More Detail

For organisations with strict data governance requirements, local AI coding assistance is not just a preference — it may be a compliance requirement. Source code often contains trade secrets, proprietary algorithms, internal API designs, and architectural decisions that represent significant competitive value. Sending this code to a third-party API means it transits external infrastructure, is processed on external servers, and may be subject to that provider’s data retention policies regardless of what their terms of service say about training data. Local models eliminate this entirely: the code never leaves your machine or your network. For financial services, healthcare, defence contractors, and any organisation handling regulated data, this is a meaningful compliance and risk management consideration that justifies the quality tradeoff of local models for at least the most sensitive parts of the codebase.

Leave a Comment