How to Run Ollama on WSL2 (Windows Subsystem for Linux)

WSL2 (Windows Subsystem for Linux 2) gives you a full Linux kernel running inside Windows — and Ollama on WSL2 is a genuinely good setup. You get native Linux performance, GPU passthrough for NVIDIA cards, and the ability to run Ollama alongside your Linux development toolchain without leaving Windows. If you’re a developer who primarily works in WSL2 but wants local LLM access, this guide is for you.

There are two main approaches: running Ollama natively inside WSL2, or running Ollama on Windows and accessing it from WSL2. Both are useful. We’ll cover both and when to choose each.

Option 1: Running Ollama Natively Inside WSL2

This is the cleaner setup for most developers. Ollama runs entirely within the Linux environment, and you interact with it from your WSL2 terminal just like you would on a native Linux machine.

First, make sure you have WSL2 installed and set as default. In a Windows PowerShell (as Administrator):

wsl --install
wsl --set-default-version 2

Then open your WSL2 terminal (Ubuntu 24.04 is recommended) and install Ollama the same way as on native Linux:

curl -fsSL https://ollama.com/install.sh | sh

The install script detects the Linux environment inside WSL2 and installs appropriately. Once installed:

ollama serve &   # start in background
ollama run llama3.2

Or start the server in one terminal and use the CLI in another — same workflow as any Linux installation.

Enabling GPU Acceleration in WSL2

WSL2 supports NVIDIA GPU passthrough via CUDA, and Ollama takes full advantage of it. This is one of the genuinely impressive things about the modern WSL2 setup — you get near-native GPU performance inside the Linux environment.

Prerequisites on the Windows side:

1. Windows 11 (or Windows 10 21H2+)
2. NVIDIA driver version 510+ installed on Windows (not inside WSL — the Windows driver is what WSL2 uses)

Inside WSL2, verify CUDA is available:

nvidia-smi

If this shows your GPU and CUDA version, you’re ready. Ollama will automatically detect and use the GPU when running models. You should not need to install CUDA separately inside WSL2 — the GPU driver is passed through from Windows.

If nvidia-smi fails inside WSL2, check that your Windows NVIDIA driver is up to date (download from nvidia.com, not through GeForce Experience). Driver 510+ is required for WSL2 GPU passthrough. After updating the Windows driver, WSL2 should pick it up on the next WSL restart:

wsl --shutdown   # run this in Windows PowerShell
# then reopen your WSL2 terminal

Option 2: Accessing Windows Ollama from WSL2

If you already have Ollama running on Windows and just want to access it from WSL2 — for example, to use it from a Python script running in your Linux environment — you don’t need to install Ollama inside WSL2 at all.

WSL2 can reach Windows services via the special hostname host.docker.internal or directly via the Windows host IP. Find your Windows host IP from inside WSL2:

cat /etc/resolv.conf | grep nameserver | awk '{print $2}'

Then set Ollama’s host to point at Windows, where Ollama is running on port 11434. In your WSL2 terminal:

export OLLAMA_HOST=http://$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):11434
ollama list   # should show models from your Windows Ollama install

For this to work, Ollama on Windows must be configured to listen on all interfaces (OLLAMA_HOST=0.0.0.0:11434) rather than just localhost. Set this in Windows Environment Variables, then restart Ollama from the system tray.

This option is useful if you want GPU acceleration on Windows (which is generally better supported) while accessing the API from Linux tooling. The downside is you need Ollama running on both sides of the fence — or at least the Windows instance managing the models.

Persisting the Ollama Service in WSL2

By default, WSL2 doesn’t run systemd, which means services don’t start automatically. There are two approaches to keeping Ollama running persistently in WSL2.

Modern WSL2 with systemd enabled (recommended): Recent WSL2 versions support systemd. Enable it by adding this to /etc/wsl.conf inside your WSL2 instance:

sudo nano /etc/wsl.conf

[boot]
systemd=true

Restart WSL2 from PowerShell:

wsl --shutdown

After restarting, systemd will be active and you can manage Ollama exactly as you would on a native Linux machine:

sudo systemctl enable ollama
sudo systemctl start ollama
systemctl status ollama

Without systemd (older WSL2 or preference): Add Ollama to your shell’s startup. Add this to your ~/.bashrc or ~/.zshrc:

# Start Ollama if not already running
if ! pgrep -x "ollama" > /dev/null; then
    ollama serve > /dev/null 2>&1 &
fi

This starts Ollama in the background whenever you open a WSL2 terminal, if it’s not already running.

Configuring Ollama in WSL2

Environment variables work the same in WSL2 as on native Linux. Set them in your ~/.bashrc for shell-based starts, or in the systemd service override if using systemd:

# In ~/.bashrc (for non-systemd setup)
export OLLAMA_MODELS=/mnt/d/ollama-models    # store models on Windows D: drive
export OLLAMA_HOST=0.0.0.0:11434              # expose to network
export OLLAMA_KEEP_ALIVE=10m

Note the /mnt/d/ path — WSL2 mounts your Windows drives under /mnt/. Storing models on your Windows D: or E: drive (rather than inside the WSL2 filesystem) is useful if you want to share them between the WSL2 and Windows Ollama installations, or if your WSL2 virtual disk is size-constrained.

Using Ollama from Python in WSL2

With Ollama running inside WSL2, you can use it from any Linux tool — Python scripts, shell scripts, Docker containers running in WSL2. Install the Python library:

pip install ollama

import ollama

response = ollama.generate(
    model='llama3.2',
    prompt='Explain what WSL2 is in two sentences.'
)
print(response['response'])

This works the same as on native Linux. The Ollama library connects to localhost:11434 by default, which hits the Ollama server running in your WSL2 instance.

Troubleshooting WSL2-Specific Issues

nvidia-smi works but Ollama isn’t using GPU. This can happen if the CUDA libraries inside WSL2 are incomplete. The install script usually handles this, but if not: sudo apt install nvidia-cuda-toolkit inside WSL2 sometimes resolves it. Alternatively, reinstall Ollama — the script checks for CUDA on install and configures accordingly.

Ollama listening on localhost but not reachable from Windows. WSL2 and Windows use separate network namespaces. To reach WSL2 services from Windows, you need the WSL2 instance’s IP, not localhost. Run ip addr show eth0 inside WSL2 to find it, or use localhost if you’ve set up WSL2 localhost forwarding in your Windows settings.

Models stored on Windows drive are slow. Cross-filesystem access (WSL2 accessing /mnt/c/ or Windows accessing WSL2 files) has significant I/O overhead. For best performance, store Ollama models inside the WSL2 filesystem rather than on a mounted Windows drive. Use a Windows drive only if you genuinely need to share models between both environments.

WSL2 runs out of memory for large models. WSL2 limits memory by default to 50% of your Windows RAM (or 8GB, whichever is less). For large models, create or edit ~/.wslconfig in your Windows user profile to increase the limit:

[wsl2]
memory=32GB
processors=8

Restart WSL2 after changing this. Without it, you’ll hit out-of-memory errors on 13B+ models even if your machine has plenty of physical RAM.

WSL2 is a great environment for Ollama if you’re already living in a Linux development workflow on Windows. The GPU passthrough works reliably on modern hardware, the systemd integration makes it easy to manage as a service, and being able to use the Ollama API from the same terminal as your other Linux tools keeps everything in one place.

WSL2 vs Native Linux for Ollama: Which Is Actually Better?

If you have the choice between running Ollama on a native Linux machine and running it in WSL2, native Linux is marginally better — slightly less overhead, simpler networking, and no memory limits to configure. But the gap is smaller than you might expect. WSL2 GPU passthrough delivers near-native CUDA performance, and for most workloads the difference in tokens per second is within the margin of measurement error. The real question is whether you’re already working in WSL2 for development. If you are, running Ollama there too keeps everything in one environment and avoids the friction of switching between Windows and Linux contexts. If you’re not already in WSL2 and you’re primarily using Ollama from Windows applications, the native Windows Ollama install is simpler and more straightforward. WSL2 shines specifically for the developer who lives in a Linux terminal on a Windows machine and wants their local LLM accessible from that same context without jumping through hoops.

Performance Expectations

With GPU passthrough enabled and a recent NVIDIA card, performance inside WSL2 is close to native Linux — typically within 5–10% on tokens per second for most models. The main performance difference is disk I/O for loading large models: loading from the WSL2 virtual disk is slightly slower than native Linux disk access, though the difference is only noticeable at load time, not during inference. A 7B model loading in 8 seconds on native Linux might take 10–12 seconds in WSL2. Once the model is loaded and generating, you’re at near-native speed. If you find load times are slow, store models inside the WSL2 filesystem rather than on a mounted Windows drive — cross-filesystem access is the main I/O bottleneck, not WSL2 overhead itself.

Setting up WSL2 with Ollama takes about 15 minutes start to finish — installing WSL2, enabling systemd, installing Ollama, confirming GPU passthrough — and once it’s working it’s a stable, low-maintenance setup that stays out of your way while keeping a capable local LLM one terminal command away.

What to Set Up Next

Once Ollama is running in WSL2, the natural next step is connecting it to your editor. The Continue extension in VS Code works with Ollama on WSL2 — point it at http://localhost:11434 and select your model. Since VS Code has native WSL2 support (the Remote – WSL extension), your editor and your local LLM backend are running in the same Linux environment, which keeps the integration clean. If you run Docker containers in WSL2, Open WebUI is a one-command addition that gives you a browser-based chat UI without any Windows-side configuration. And if you write Python scripts in WSL2, the Ollama Python library connects to the local server the same way it would on native Linux — no special configuration needed beyond making sure the server is running before your script starts.