How to Install Ollama on Windows 11 (Complete Setup Guide)

by

Running large language models locally on Windows used to be a painful experience — CUDA drivers, Python environments, mysterious errors. Ollama changed that. It wraps everything into a single installer and gives you a clean CLI and API to pull and run models in minutes. This guide walks you through installing Ollama on Windows 11, getting your first model running, and setting it up properly so it actually works the way you want — including GPU acceleration, network access, and running it as a background service.

No prior experience with local LLMs required. If you’ve got a Windows 11 machine and a bit of patience, you’ll have a model running by the end of this.

What You Need Before You Start

Ollama on Windows is straightforward, but a few things are worth checking upfront so you don’t hit surprises halfway through.

Operating system: Windows 11 is fully supported. Windows 10 (version 1903 or later) also works, but this guide focuses on Windows 11. If you’re on an older version, upgrade first.

RAM: This is the biggest constraint. The model has to fit in memory at runtime. As a rough guide — 8GB RAM lets you run 3B–7B models comfortably. 16GB opens up 13B models. 32GB or more and you can run 30B–70B models, though you’ll need patience. If you’re tight on RAM, smaller quantized models (Q4 format) are your friend and quality is often surprisingly good.

GPU (optional but recommended): Ollama on Windows supports NVIDIA GPUs via CUDA. If you have an NVIDIA card with at least 4GB VRAM, Ollama will automatically use it for inference — which is dramatically faster than CPU-only. AMD GPU support on Windows is more limited; CPU offloading is the safer bet for AMD users for now.

Disk space: Models are large. A 7B model in Q4 format is around 4–5GB. A 13B model is around 8GB. Make sure the drive where Ollama stores models has enough headroom — by default it uses your user profile directory.

Internet connection: You only need it to download the installer and pull models. Once a model is downloaded, everything runs offline.

Downloading and Installing Ollama

Head to ollama.com and click the Download button. It detects your OS and serves the Windows installer directly — a standard .exe file, no fuss.

Run the installer. It doesn’t ask many questions — it installs Ollama, sets up the background service, and adds the ollama command to your PATH automatically. The whole thing takes under a minute on most machines.

Once the installer finishes, open a new terminal window. PowerShell or Windows Terminal both work fine. Run:

ollama --version

You should see something like ollama version 0.x.x. If you get “command not found”, close and reopen your terminal — the PATH update from the installer sometimes needs a fresh session to take effect.

Ollama runs as a background Windows service after installation. You’ll see the Ollama icon in your system tray. It starts automatically on boot, which means it’s always ready when you need it without you having to remember to launch it manually.

Running Your First Model

With Ollama installed, you’re one command away from a running model. Let’s start with Llama 3.2 3B — small enough to run on most machines, capable enough to be genuinely useful:

ollama run llama3.2

The first time you run this, Ollama downloads the model from its registry. The 3B model is around 2GB, so download time depends on your connection. Watch the progress bar — once it hits 100%, the model loads and you’ll see a >>> prompt. Type anything and it responds.

To exit the chat, type /bye or hit Ctrl+D.

If you want a bigger model and have the RAM for it, swap in a different model name:

ollama run llama3.1:8b
ollama run mistral
ollama run phi4

You can browse the full library at ollama.com/library. Every model page shows the available sizes and their download sizes so you can pick what fits your machine.

Enabling GPU Acceleration on NVIDIA Cards

If you have an NVIDIA GPU, Ollama should detect and use it automatically. You can verify this by watching the output when a model first loads — you’ll see a line indicating whether it’s using GPU layers or CPU-only. To check explicitly:

ollama run llama3.2 --verbose

Look for output mentioning gpu_layers. If it shows a non-zero number, your GPU is being used. If it shows 0, Ollama is running on CPU.

If your GPU isn’t being picked up, the most common cause is a missing or outdated CUDA driver. Ollama requires CUDA 11.3 or later. Download the latest driver from nvidia.com/drivers — not just a GeForce Experience update, but the full driver package. After installing, restart your machine and try again.

One thing to be aware of: Ollama uses your GPU VRAM to load model layers. If the model is larger than your VRAM, it splits — some layers go on GPU, the rest on CPU RAM. This is called partial GPU offloading and it still gives you a speed improvement over pure CPU, though not as much as a full GPU load. Ollama handles this automatically, so you don’t need to configure anything.

If you want to force CPU-only mode for testing (or because your GPU is being used by something else), set this environment variable before running:

set CUDA_VISIBLE_DEVICES=-1
ollama run llama3.2

Changing Where Ollama Stores Models

By default, Ollama stores downloaded models in your user profile directory — something like C:UsersYourName.ollamamodels. If your C: drive is tight on space, you’ll want to move this somewhere else.

Set the OLLAMA_MODELS environment variable to point to your preferred location. The easiest way to do this persistently on Windows is through System Properties:

  1. Search for “Environment Variables” in the Start menu and open “Edit the system environment variables”
  2. Click “Environment Variables” at the bottom
  3. Under “User variables”, click New
  4. Variable name: OLLAMA_MODELS
  5. Variable value: the full path to your desired folder, e.g. D:ollama-models
  6. Click OK and restart your terminal

Any models you pull after this point will go to the new location. Models you already downloaded stay where they are — you can move them manually to the new folder if you want to consolidate.

Another useful variable is OLLAMA_HOST, which controls the address and port Ollama’s API server listens on. By default it’s 127.0.0.1:11434 — accessible only from your own machine. We’ll cover opening this up to your local network in a moment.

Figure 1 — Ollama Windows: RAM & VRAM Requirements by Model Size

Model Size Format Min RAM GPU VRAM Speed (CPU) 3B (e.g. Llama 3.2 3B)Q44 GB4 GB+Fast 7B (e.g. Llama 3.1 8B)Q48 GB6 GB+OK 13B (e.g. Mistral 7B Q8)Q4/Q816 GB10 GB+Slow 34B (e.g. CodeLlama 34B)Q424 GB20 GB+Very slow 70B (e.g. Llama 3.1 70B)Q440 GB+48 GB+CPU: painful Q4 quantization recommended for most users — excellent quality/size trade-off

Accessing Ollama from Other Devices on Your Network

By default Ollama only accepts connections from your own machine. If you want to use it from another device — say, querying it from your laptop while it runs on your desktop, or connecting Open WebUI running in Docker — you need to open it up.

Add a new user environment variable:

  • Variable name: OLLAMA_HOST
  • Variable value: 0.0.0.0:11434

Then restart the Ollama service. Right-click the Ollama icon in your system tray and choose Quit, then launch Ollama again from the Start menu. With 0.0.0.0, Ollama will accept connections on port 11434 from any device on your network.

You’ll also want to allow this through Windows Firewall. Search for “Windows Defender Firewall” in the Start menu, go to Advanced Settings, create a new Inbound Rule for TCP port 11434, and allow the connection. Without this, the firewall will silently block incoming requests even if Ollama is listening.

Test it from another machine using curl or just your browser — navigate to http://[your-windows-PC-IP]:11434 and you should see Ollama is running.

Using the Ollama API

Once Ollama is running, it exposes a REST API at http://localhost:11434. This is what tools like Open WebUI, Continue, and Cursor use under the hood — but you can hit it directly too.

A quick test with curl to generate a response:

curl http://localhost:11434/api/generate -d "{"model":"llama3.2","prompt":"Why is the sky blue?","stream":false}"

Or from Python using the requests library:

import requests

response = requests.post(
    "http://localhost:11434/api/generate",
    json={
        "model": "llama3.2",
        "prompt": "Explain gradient descent in one paragraph.",
        "stream": False
    }
)
print(response.json()["response"])

Ollama also supports an OpenAI-compatible endpoint at /v1/chat/completions, which means you can drop it in as a replacement for the OpenAI API in any tool that supports custom base URLs — just point your client at http://localhost:11434/v1 and set any non-empty string as the API key.

Useful Ollama Commands to Know

A few commands you’ll use regularly once you’re set up:

# List all downloaded models
ollama list

# Pull a model without running it
ollama pull mistral

# Remove a model you no longer need
ollama rm llama3.2

# Show model details and parameters
ollama show llama3.1:8b

# Check what's currently running
ollama ps

# Run a model with a custom system prompt
ollama run llama3.2 --system "You are a concise assistant. Always answer in bullet points."

The ollama ps command is handy for debugging — it shows which model is loaded in memory, how much VRAM it’s using, and when it was last used. By default, Ollama keeps a model loaded for 5 minutes after your last request before unloading it. You can change this with the OLLAMA_KEEP_ALIVE environment variable (e.g., set it to 0 to unload immediately, or -1 to keep it loaded indefinitely).

Figure 2 — Ollama Windows: Common Environment Variables

Variable Default What it does OLLAMA_MODELS~/.ollama/modelsWhere models are stored OLLAMA_HOST127.0.0.1:11434API listen address and port OLLAMA_KEEP_ALIVE5mHow long to keep model in memory OLLAMA_NUM_PARALLEL1Parallel request handling CUDA_VISIBLE_DEVICESall GPUsSet to -1 to force CPU mode

Troubleshooting Common Issues

“ollama” is not recognized as a command. The installer adds Ollama to your PATH, but the terminal needs to be reopened to pick up the change. Close and reopen PowerShell or Windows Terminal. If it still doesn’t work, check that C:UsersYourNameAppDataLocalProgramsOllama exists and add it to your PATH manually via Environment Variables.

Model download stalls or fails. Check your internet connection and try again. If a partial download caused the issue, delete the incomplete model file from your models directory and re-run ollama pull. You can also try setting OLLAMA_DEBUG=1 to get verbose output and see exactly where it’s getting stuck.

GPU not being used. First, check your NVIDIA driver version — run nvidia-smi in a terminal and confirm CUDA is available. If nvidia-smi itself fails, you need to reinstall your GPU driver. If CUDA is showing but Ollama still isn’t using it, try restarting the Ollama service from the system tray.

Out of memory errors. The model is too large for your available RAM or VRAM. Try a smaller quantized version — most models in the Ollama library have Q4 variants that are half the size with minimal quality loss. Run ollama pull llama3.1:8b-q4_0 instead of the default tag to get the Q4 version explicitly.

Ollama service won’t start. Check Windows Event Viewer for errors (search “Event Viewer” in Start, look under Windows Logs → Application). Port 11434 might be in use by another process — run netstat -ano | findstr 11434 to check. If something else is using that port, change Ollama’s port via the OLLAMA_HOST variable.

What to Try Next

Once Ollama is running and you’ve got a model chatting in your terminal, the obvious next step is adding a proper UI. Open WebUI is the most polished option — it’s a web interface that connects to your local Ollama instance and gives you a ChatGPT-style experience, with conversation history, model switching, and file uploads. It runs in Docker, so you’ll need Docker Desktop installed, but setup is a single command.

If you’re a developer, connecting Ollama to your IDE is worth doing early. The Continue extension for VS Code and Cursor both support Ollama as a backend — you get AI-assisted coding with zero API costs and nothing leaving your machine. For anyone working with sensitive code or data, that’s a genuine advantage over cloud-hosted alternatives.

The model library keeps growing too. Worth checking out Qwen 2.5 Coder for code tasks, Phi-4 for a surprisingly capable small model, and DeepSeek R1 distilled models if you want reasoning-style outputs on consumer hardware. Run ollama pull with any of those names and you’re running them in minutes.

Leave a Comment