Ollama has a large set of parameters that control how models behave, how much memory they use, and how they run on your hardware. Most users never touch these and get reasonable results — but knowing what’s available and what each setting does unlocks significantly better outcomes for specific use cases. This guide covers every parameter worth knowing, when to change it, and what to expect when you do.
How to Set Parameters in Ollama
Parameters can be set in three places, each with different scope:
CLI flag (session only):
ollama run llama3.2 --parameter temperature 0.2
ollama run llama3.2 --parameter num_ctx 16384 --parameter temperature 0.1Inside a chat session:
>>> /set parameter temperature 0.5
>>> /set parameter num_ctx 8192Modelfile (persistent per-model config):
cat > Modelfile << 'EOF'
FROM llama3.2
PARAMETER temperature 0.3
PARAMETER num_ctx 16384
PARAMETER top_p 0.9
SYSTEM "You are a precise technical assistant."
EOF
ollama create my-assistant -f ModelfileVia the API (per-request):
import ollama
response = ollama.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Hello'}],
options={
'temperature': 0.2,
'num_ctx': 8192,
'top_p': 0.9,
}
)Temperature: Creativity vs Precision
Temperature is the single most impactful parameter for output character. It controls the randomness of token selection — at temperature 0, the model always picks the most probable next token; at high temperatures, it samples more broadly from the distribution.
0.0 – 0.2: Highly deterministic. Near-identical outputs for identical inputs. Best for: structured data extraction, JSON generation, factual Q&A where you want consistent answers, code generation where correctness matters more than creativity. The output is precise and repeatable but can feel mechanical.
0.3 – 0.6: Balanced. Good mix of coherence and variation. Best for: technical writing, documentation, explaining concepts, most everyday assistant tasks. The default for most models is around 0.8, but 0.4–0.5 often produces cleaner outputs for analytical work.
0.7 – 1.0: Creative and varied. Noticeably different outputs each run. Best for: creative writing, brainstorming, generating diverse alternatives, poetry, fiction. The default range for most models.
Above 1.0: Increasingly random. Can produce interesting creative outputs but also nonsense. Generally only useful for very specific creative applications.
# Precise code generation
ollama run qwen2.5-coder:7b --parameter temperature 0.1
# Creative writing
ollama run llama3.2 --parameter temperature 0.9
# Balanced general use
ollama run llama3.2 --parameter temperature 0.4num_ctx: Context Window
The context window determines how many tokens the model can see at once — your system prompt, the full conversation history, and the response being generated all count toward this limit. Ollama defaults to 2048 regardless of model capability, which truncates conversations prematurely for most modern models that support 128K.
# For casual chat — 4096 covers most conversations
ollama run llama3.2 --parameter num_ctx 4096
# For coding sessions with large files
ollama run qwen2.5-coder:7b --parameter num_ctx 16384
# For document analysis
ollama run llama3.1:8b --parameter num_ctx 32768The memory cost of context scales with model size — a 7B model needs roughly 0.5–0.8GB extra per 8K tokens of context, a 70B model needs 5GB+ per 8K tokens. Check your VRAM headroom before setting very high contexts on large models. When context fills up, Ollama silently drops the oldest tokens — the model "forgets" the beginning of the conversation. Check context usage with ollama ps during a long session.
top_p and top_k: Sampling Controls
These work alongside temperature to shape how tokens are sampled:
top_p (nucleus sampling, default ~0.9): Consider only the smallest set of tokens whose cumulative probability exceeds top_p. At 0.9, the model samples from tokens that together account for 90% of probability mass — filtering out very unlikely tokens while preserving reasonable variation. Lowering to 0.7 makes outputs more focused; raising to 0.95 allows more diverse vocabulary.
top_k (default ~40): Consider only the top K most probable tokens at each step. At 40, only the 40 most likely next tokens are considered. Lower values (10–20) produce more focused output; higher values (60–100) allow more vocabulary breadth. Works in combination with top_p — both filters are applied.
# Focused, controlled output
ollama run llama3.2 --parameter top_p 0.7 --parameter top_k 20
# More diverse, creative output
ollama run llama3.2 --parameter top_p 0.95 --parameter top_k 60For most use cases, the default top_p and top_k values work well. If you notice the model repeating words or phrases unnaturally, try lowering top_k. If outputs feel too constrained or repetitive at the sentence level, raise top_p slightly.
repeat_penalty: Reducing Repetition
repeat_penalty discourages the model from repeating tokens it recently generated. The default is usually 1.1 — slightly penalising repetition. Increasing it reduces repetitive output; setting it to 1.0 disables the penalty entirely.
# Reduce repetitive lists or paragraphs
ollama run llama3.2 --parameter repeat_penalty 1.2
# Disable penalty (useful when repetition is correct, e.g. code patterns)
ollama run llama3.2 --parameter repeat_penalty 1.0If your model produces outputs that repeat the same phrase or structure multiple times, increase repeat_penalty to 1.15–1.3. For code generation where the same pattern legitimately repeats (e.g. multiple similar function definitions), set it to 1.0 to avoid the model artificially varying syntax that should be consistent.
Figure 1 — Ollama Parameters Quick Reference
num_predict: Controlling Output Length
num_predict caps the number of tokens the model can generate in a single response. The default is -1 (unlimited — the model generates until it decides to stop). For applications where you need predictable, bounded output lengths, setting this is important:
# Short answers only
ollama run llama3.2 --parameter num_predict 100
# Medium responses
ollama run llama3.2 --parameter num_predict 500
# Long documents allowed
ollama run llama3.2 --parameter num_predict 2000For batch processing where you're generating many items and need consistent timing, setting num_predict prevents any single item from generating an unexpectedly long response that holds up the rest of the batch. For interactive chat, leaving it at -1 lets the model determine appropriate length — most models stop naturally at the end of a complete answer.
seed: Reproducible Outputs
Setting a seed makes generation deterministic — the same prompt with the same seed at the same temperature produces the same output every time. Essential for testing and evaluation:
ollama run llama3.2 --parameter seed 42 --parameter temperature 0Use seed=42 (or any fixed integer) combined with temperature=0 for fully reproducible outputs. Useful when benchmarking models, comparing prompts, or building tests where you need consistent baseline outputs to detect regressions. Note: reproducibility is hardware-dependent — the same seed may produce slightly different outputs on different GPU models or CPU vs GPU inference.
GPU vs CPU Mode: How to Switch
Ollama automatically detects and uses your GPU. To force CPU-only mode (for testing, debugging, or running on hardware without a GPU):
# Force CPU on NVIDIA systems
CUDA_VISIBLE_DEVICES="" ollama run llama3.2
# Force CPU on any system
OLLAMA_HOST=127.0.0.1:11434 ollama run llama3.2 # then in API call:
# options: {"num_gpu": 0}
Via the API, set num_gpu: 0 to force CPU inference for a specific request, or num_gpu: 99 to put as many layers on GPU as possible (Ollama caps at the actual layer count). The num_gpu parameter specifically controls how many transformer layers run on GPU — setting it to a value between 0 and the model's total layer count achieves partial GPU offloading:
import ollama
# Full GPU (default behaviour)
response = ollama.chat(model='llama3.2', messages=[...], options={'num_gpu': 99})
# CPU only
response = ollama.chat(model='llama3.2', messages=[...], options={'num_gpu': 0})
# Partial — first 20 layers on GPU, rest on CPU
response = ollama.chat(model='llama3.2', messages=[...], options={'num_gpu': 20})Partial GPU offloading is useful when the model slightly exceeds your VRAM. Putting as many layers as fit in VRAM (while running the rest on CPU RAM) gives meaningfully faster inference than pure CPU even if not all layers fit on GPU. Experiment with num_gpu values to find the highest value that doesn't cause VRAM OOM errors on your hardware.
Reducing Memory Usage
When memory is tight, several settings reduce Ollama's footprint without switching to a completely different model:
Lower quantization: The most effective memory reduction. Switching from Q4_K_M to Q4_0 saves 10–15%; switching to Q2_K halves the model size at meaningful quality cost.
Reduce num_ctx: Every 8K of context adds 0.5–5GB to memory depending on model size. Setting num_ctx to 2048 (Ollama's default) uses minimal KV cache memory.
Enable Flash Attention: Reduces KV cache memory usage by 30–50% at long context lengths. Set OLLAMA_FLASH_ATTENTION=1.
Reduce num_gpu: Offloading more layers to CPU RAM (which is typically larger than VRAM) at the cost of inference speed.
# Memory-optimised run
export OLLAMA_FLASH_ATTENTION=1
ollama run llama3.1:8b-instruct-q4_0 --parameter num_ctx 2048 --parameter num_gpu 20 # adjust to fit your VRAMFigure 2 — Recommended Parameter Presets by Use Case
Mirostat: Adaptive Sampling
Mirostat is an alternative sampling algorithm that dynamically adjusts temperature during generation to maintain a target perplexity — producing outputs that are neither too predictable nor too chaotic, regardless of the starting temperature setting. Two versions are available:
# Mirostat v1 — simpler, good for short texts
ollama run llama3.2 --parameter mirostat 1 --parameter mirostat_tau 5.0
# Mirostat v2 — recommended, better for long generations
ollama run llama3.2 --parameter mirostat 2 --parameter mirostat_tau 5.0mirostat_tau controls the target entropy — lower values (3.0–4.0) produce more focused output, higher values (6.0–8.0) produce more varied output. When mirostat is enabled, temperature and top_p/top_k are largely overridden by the adaptive algorithm. Mirostat is particularly useful for long-form creative writing and extended conversations where standard temperature settings can cause the model to "drift" into repetitive or incoherent patterns over many tokens. For shorter, task-focused generation, standard temperature + top_p is usually more predictable.
Environment Variables: Ollama-Level Settings
Some settings apply to the Ollama server itself rather than individual model runs. These are set as environment variables before starting Ollama:
OLLAMA_FLASH_ATTENTION=1 # Enable Flash Attention (recommended)
OLLAMA_KEEP_ALIVE=30m # How long to keep models loaded (default 5m)
OLLAMA_NUM_PARALLEL=2 # Concurrent requests (default 1)
OLLAMA_MAX_LOADED_MODELS=2 # Max models in memory simultaneously
OLLAMA_HOST=0.0.0.0:11434 # Listen on all interfaces (not just localhost)
OLLAMA_ORIGINS=* # Allow CORS from any origin (for browser access)
OLLAMA_MODELS=/custom/path # Custom models directoryAdd these to your shell profile (~/.bashrc, ~/.zshrc, or Windows environment variables) to persist them across restarts. On Linux with systemd, add them to the service override file:
sudo systemctl edit ollama
# Add:
[Service]
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KEEP_ALIVE=1h"Finding Your Optimal Settings
The right parameter combination is task-specific and model-specific — there's no universal "best" setting. The most effective approach is systematic experimentation: pick one task you run frequently, fix all other variables, and test two or three values for a single parameter. Compare outputs side by side. The temperature parameter almost always has the most impact, so start there. Once you've found a good temperature, experiment with top_p and repeat_penalty. Only venture into more obscure parameters (mirostat, tfs_z, typical_p) if the standard parameters aren't giving you what you need — for most use cases, temperature and num_ctx are the only settings that matter. Once you've found a configuration that works well, save it as a Modelfile so it applies automatically every time you use that model, without having to re-specify flags each session.
Advanced Sampling: tfs_z and typical_p
Two less commonly used but occasionally useful sampling methods:
tfs_z (tail free sampling): Removes tokens from the tail of the probability distribution that are unlikely to be coherent continuations. Values between 0.9 and 1.0 are typical — 1.0 disables it. Useful when the model occasionally generates very unlikely tokens (names, technical terms) that feel out of place. Setting tfs_z to 0.95 quietly trims these without significantly constraining the vocabulary for normal tokens. Most users don't need this, but if you notice occasional bizarre word choices in otherwise coherent text, it's worth trying.
typical_p (locally typical sampling): An alternative to nucleus sampling that selects tokens based on their information content relative to the conditional entropy rather than their raw probability. More theoretically motivated than top_p and tends to produce more natural-sounding text, though the difference is subtle in practice. Values around 0.9–0.95 are common when used. Both tfs_z and typical_p can be combined with standard temperature and top_p settings — they apply as additional filters on the sampling distribution. Unless you're actively researching sampling strategies, start with temperature and top_p, and only reach for these if you're looking for subtle improvements to text naturalness after the main parameters are tuned.
Using Modelfiles to Lock in Your Best Settings
Once you've found a parameter combination that works well for your workflow, save it in a Modelfile so it applies automatically rather than needing to be specified as flags every time. This is particularly valuable if you use different configurations for different types of work — a code-focused configuration and a writing-focused configuration for the same base model:
cat > Modelfile-code << 'EOF'
FROM qwen2.5-coder:7b
PARAMETER temperature 0.1
PARAMETER num_ctx 16384
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.0
SYSTEM "You are an expert software engineer. Write clean, production-quality code with appropriate error handling and documentation."
EOF
ollama create code-assistant -f Modelfile-code
cat > Modelfile-write << 'EOF'
FROM llama3.1:8b
PARAMETER temperature 0.7
PARAMETER num_ctx 8192
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.15
SYSTEM "You are a skilled technical writer. Produce clear, engaging prose that's accurate and well-structured."
EOF
ollama create writing-assistant -f Modelfile-writeNow ollama run code-assistant and ollama run writing-assistant both use your tuned parameters automatically, visible in ollama list, callable from any tool that uses Ollama. This approach — building a small library of purpose-tuned model variants from the same base models — is the right pattern for power users who work across multiple task types daily.