Local LLMs have become genuinely useful coding assistants. With the right setup, you can have AI-powered autocomplete, inline chat, code explanation, and refactoring suggestions running entirely on your own hardware — no API costs, no rate limits, no data leaving your machine. This guide covers the practical setup for using Ollama as a coding backend with VS Code, plus direct Python workflows for code generation and review.
Choosing the Right Coding Model
Not all local LLMs perform equally on code. Models specifically trained on large code corpora outperform general-purpose models significantly on coding tasks — better completion quality, more accurate API usage, fewer hallucinated method names, and stronger understanding of context across multiple files. The top choices from the Ollama library in 2026:
Qwen 2.5 Coder (7B and 14B): The strongest coding models available locally. Exceptional performance on code completion, debugging, and explanation across Python, JavaScript, TypeScript, Go, Rust, Java, and other major languages. The 7B version is fast enough for real-time autocomplete on most GPUs. The 14B version produces noticeably better outputs on complex tasks if your hardware supports it.
DeepSeek Coder V2 (16B): Strong on algorithmic problems, competitive programming, and code that requires deep reasoning. Slower than Qwen 2.5 Coder 7B but competitive on quality.
Llama 3.1 (8B, 70B): Good general coding ability. Not as strong as dedicated coding models on code-specific tasks but excellent if you want one model that handles both coding and general tasks.
Phi-4 (14B): Surprisingly strong on algorithmic reasoning and math-heavy code. Smaller footprint than DeepSeek, good quality on structured programming tasks.
ollama pull qwen2.5-coder:7b # best speed/quality balance
ollama pull qwen2.5-coder:14b # better quality, needs more VRAM
ollama pull deepseek-coder-v2 # strong on algorithmsVS Code Integration: Continue Extension
Continue is the most capable VS Code extension for local AI coding assistance. It integrates Ollama as a backend and provides inline autocomplete, a chat sidebar, inline code editing, and codebase indexing. Install it from the VS Code marketplace:
code --install-extension Continue.continueAfter installation, open the Continue sidebar (the Continue icon in the left panel) and configure it to use Ollama. Create or edit ~/.continue/config.json:
{
"models": [
{
"title": "Qwen 2.5 Coder 7B",
"provider": "ollama",
"model": "qwen2.5-coder:7b",
"apiBase": "http://localhost:11434"
},
{
"title": "Llama 3.1 8B (general)",
"provider": "ollama",
"model": "llama3.1",
"apiBase": "http://localhost:11434"
}
],
"tabAutocompleteModel": {
"title": "Autocomplete",
"provider": "ollama",
"model": "qwen2.5-coder:7b",
"apiBase": "http://localhost:11434"
},
"embeddingsProvider": {
"provider": "ollama",
"model": "nomic-embed-text",
"apiBase": "http://localhost:11434"
}
}With this config, Continue uses Qwen 2.5 Coder for both inline autocomplete and the chat sidebar. The embedding model enables codebase indexing — Continue can index your entire project and retrieve relevant context when answering questions about your code.
Using Continue’s Key Features
Inline autocomplete: Works like GitHub Copilot. Start typing and suggestions appear greyed out — Tab to accept, Escape to dismiss. The autocomplete model runs continuously in the background, sending your current file context to Ollama for completions. Response speed depends on your hardware; on a GPU with Qwen 2.5 Coder 7B you get suggestions in 0.5-2 seconds.
Chat sidebar: Cmd/Ctrl+L opens the chat panel. Ask questions about your code, request refactors, explain functions. Highlight code first and press Cmd/Ctrl+L to include it as context automatically.
Inline editing: Highlight code and press Cmd/Ctrl+I to open an inline edit prompt. Type what you want changed — “add error handling”, “convert to async”, “add type annotations” — and Continue applies the change directly in your editor with a diff view to accept or reject.
Codebase context (@codebase): Type @codebase in the chat to enable retrieval from your indexed project. Ask “how is authentication handled in this codebase?” and Continue retrieves relevant files before answering.
Figure 1 — Local Coding Model Comparison
Python Workflow: Code Generation with Ollama
For scripted code generation tasks — generating boilerplate, converting code between languages, generating test cases — the Ollama Python library gives you full control:
import ollama
SYSTEM = """You are an expert software engineer. Write clean, production-quality code.
Include error handling, type hints (for Python), and brief docstrings.
Return only the code, no explanation unless asked."""
def generate_code(task: str, language: str = "python") -> str:
response = ollama.chat(
model="qwen2.5-coder:7b",
messages=[
{"role": "system", "content": SYSTEM},
{"role": "user", "content": f"Write {language} code to: {task}"}
],
options={"temperature": 0.1, "num_ctx": 8192}
)
return response["message"]["content"]
def explain_code(code: str) -> str:
response = ollama.chat(
model="qwen2.5-coder:7b",
messages=[
{"role": "system", "content": "Explain what this code does clearly and concisely."},
{"role": "user", "content": code}
],
options={"temperature": 0.3}
)
return response["message"]["content"]
def review_code(code: str) -> str:
response = ollama.chat(
model="qwen2.5-coder:7b",
messages=[
{"role": "system", "content": "Review this code for bugs, security issues, and improvements. Be specific and actionable."},
{"role": "user", "content": code}
],
options={"temperature": 0.2}
)
return response["message"]["content"]
# Examples
print(generate_code("parse a CSV file and return rows as dicts, handling encoding errors"))
print(review_code(open("my_script.py").read()))Temperature 0.1 for code generation keeps outputs deterministic and focused. Temperature 0 makes generation fully deterministic — useful when you need reproducible outputs for testing. Slightly higher temperature (0.3-0.5) for explanations allows more natural language variation.
Generating Unit Tests
One of the highest-value coding assistant tasks is test generation. Local models handle this well when given the source function as context:
def generate_tests(function_code: str, framework: str = "pytest") -> str:
response = ollama.chat(
model="qwen2.5-coder:7b",
messages=[
{
"role": "system",
"content": f"Write comprehensive {framework} unit tests for the given function. Cover happy path, edge cases, and error conditions. Include test for empty inputs, boundary values, and type errors where relevant."
},
{"role": "user", "content": f"Generate tests for:\n\n{function_code}"}
],
options={"temperature": 0.1, "num_ctx": 8192, "num_predict": 1000}
)
return response["message"]["content"]
# Test a function
my_function = """
def calculate_discount(price: float, discount_pct: float) -> float:
if not 0 <= discount_pct <= 100:
raise ValueError(f"Discount must be 0-100, got {discount_pct}")
return price * (1 - discount_pct / 100)
"""
print(generate_tests(my_function))Providing the actual function code in the prompt rather than just describing it produces much better tests — the model can see the exact input/output types, error conditions, and implementation details it needs to write meaningful test cases.
Other VS Code Integrations
Beyond Continue, a few other VS Code extensions work with Ollama. Ollama-copilot is a lighter alternative that focuses purely on autocomplete without the full chat sidebar. CodeGPT supports Ollama as a backend alongside cloud APIs, useful if you want to switch between local and cloud models within the same interface. Aider (used from the terminal rather than VS Code) is a powerful AI pair programmer that works with Ollama — it makes multi-file edits based on natural language instructions and commits the changes to git, giving you a full agentic coding workflow. For complex refactors that span multiple files, Aider backed by a strong local model is a compelling combination.
Practical Tips for Better Coding Assistance
A few practices that meaningfully improve local LLM coding assistance quality. Always include context. A model generating code in isolation makes assumptions about your project structure, naming conventions, and existing utilities. Paste relevant imports, type definitions, or related functions along with your request. The more context you provide, the less the model has to guess. Be specific about requirements. "Write a function to process orders" produces generic code. "Write a Python function that takes a list of Order dataclass instances, filters to those with status PENDING, groups them by customer_id, and returns a dict mapping customer_id to list of orders, sorted by created_at ascending" produces exactly what you need. Use low temperature for code. Temperature 0 to 0.1 produces consistent, focused code. Higher temperatures introduce variation that is useful for creative writing but harmful for code correctness. Verify outputs. Local models make mistakes — hallucinated method names, incorrect API signatures, subtle logic errors. Always read generated code before running it. Run tests. The local model is a powerful first draft generator, not a replacement for understanding the code you commit.
Context Window Considerations for Large Codebases
The most common limitation when using local LLMs as coding assistants is context window size. A 7B model with num_ctx=8192 can hold roughly 6,000 tokens of context — enough for a few hundred lines of code plus the prompt. For questions that require understanding multiple files or a large codebase, this is often insufficient. Two approaches address this: codebase indexing (what Continue's @codebase feature provides — retrieving the most relevant snippets rather than the entire codebase) and larger context windows on capable hardware. Models like Qwen 2.5 Coder support up to 128K context — setting num_ctx=32768 or higher and ensuring you have sufficient VRAM allows longer code sessions without the model losing earlier context. The memory cost of long context is real (a 7B model at 32K context uses roughly 3-5GB more VRAM than at 4K context) but on hardware with 16GB+ VRAM it is often worthwhile for complex coding sessions.
Offline Coding Assistance: The Privacy Advantage
Local coding assistants have a privacy advantage that matters in professional contexts. Pasting proprietary code, internal API designs, client project details, or unreleased features into GitHub Copilot or ChatGPT raises legitimate concerns — the code goes to external servers under those services' data policies. With a local Ollama coding setup, your code never leaves your machine. This is particularly relevant for: working on projects under NDA, proprietary algorithms you do not want to expose, client code covered by confidentiality agreements, or any codebase that your organisation restricts from third-party services. Local coding assistance removes the friction of deciding what code is "safe" to paste into a cloud AI — with local inference, all of it is safe.
Building a Code Review Pipeline
For teams or individuals who want systematic code review assistance, a simple pipeline that reviews every file in a git diff before committing catches a surprising number of issues. The pattern: get the diff, split it into per-file changes, send each file's changes with surrounding context to the local coding model, and collect the review comments. This runs in under a minute for a typical pull request on a 7B model and catches common problems — missing error handling, unvalidated inputs, obvious performance issues, and style inconsistencies — before code review by humans. It is not a replacement for human review but a useful first pass that lets human reviewers focus on higher-level concerns rather than catching basic issues. The output quality is consistently good enough to be worth running: the 20-second cost to review a 10-file PR catches problems that would otherwise require back-and-forth in review, making it a worthwhile addition to a local development workflow.
Getting Started: The 15-Minute Setup
The fastest path to a working local coding assistant: install Ollama, pull qwen2.5-coder:7b, install the Continue extension in VS Code, configure it with the JSON above, and open a code file. The total time from zero to working AI autocomplete and chat in your editor is fifteen minutes including model download on a fast connection. The Continue extension's tab autocomplete starts working immediately — begin typing in a function and watch suggestions appear. For the first few sessions, experiment with what kinds of prompts produce the best results for your specific languages and coding style. The model improves with better prompts: specific, context-rich requests produce code you can use with minimal editing, while vague requests produce generic boilerplate that needs significant rework. Investing thirty minutes in finding the prompt patterns that work for your workflow pays dividends across every future coding session.