Ollama exposes a clean REST API that lets you interact with any locally running model from your Python code — no paid API keys, no rate limits, nothing leaving your machine. Whether you’re building a chatbot, automating document processing, or integrating LLM capabilities into an existing application, the Ollama API is your bridge between Python and your local models. This guide covers every endpoint worth knowing, both the official ollama Python library and raw HTTP calls, streaming, error handling, and patterns for building reliable applications on top of it.
Two Ways to Call the Ollama API from Python
You have two clean options: the official ollama Python library (which wraps the REST API), or raw HTTP calls with requests or httpx. The library is simpler and handles streaming cleanly. Raw HTTP gives you more control and works when you need to use the OpenAI-compatible endpoint for drop-in compatibility.
Install the official library:
pip install ollamaOr for raw HTTP calls:
pip install requests httpx # httpx preferred for asyncGenerate: Single-Turn Text Completion
ollama.generate() is the simplest call — send a prompt, get a response. No conversation history, no system prompt required:
import ollama
response = ollama.generate(
model='llama3.2',
prompt='Explain what a transformer is in machine learning, in two paragraphs.'
)
print(response['response'])
print(f"
Tokens: {response['eval_count']} | Time: {response['eval_duration']/1e9:.2f}s")The response object includes useful metadata: eval_count (tokens generated), eval_duration (nanoseconds for generation), prompt_eval_count (tokens in the prompt), and total_duration. Use these to benchmark performance across models.
Chat: Multi-Turn Conversations
ollama.chat() is what you’ll use most — it supports system prompts, message history, and tool calling:
import ollama
messages = [
{
'role': 'system',
'content': 'You are a concise Python expert. Give direct answers with code examples.'
},
{
'role': 'user',
'content': 'How do I read a CSV file in Python and calculate column averages?'
}
]
response = ollama.chat(model='llama3.2', messages=messages)
print(response['message']['content'])
# Add the response to history for follow-up
messages.append(response['message'])
messages.append({'role': 'user', 'content': 'Now show me how to handle missing values.'})
response2 = ollama.chat(model='llama3.2', messages=messages)
print(response2['message']['content'])The key to multi-turn conversations is maintaining the messages list yourself — Ollama is stateless between calls. Each call must include the full conversation history. The pattern above (appending responses and new messages to the list) is the standard approach.
Streaming Responses
By default, Ollama waits for the full response before returning. For interactive applications, streaming prints tokens as they arrive:
import ollama
stream = ollama.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Write a short poem about local LLMs.'}],
stream=True
)
for chunk in stream:
print(chunk['message']['content'], end='', flush=True)
print() # newline after stream completesStreaming is important for user-facing applications — it makes the response feel instant rather than making the user wait for the full generation. For background processing where you just need the final result, non-streaming is fine.
Figure 1 — Ollama API: Endpoint Reference
Embeddings
Ollama can generate text embeddings — vector representations of text used for semantic search, similarity comparison, and RAG pipelines:
import ollama
import numpy as np
# Generate an embedding for a single text
response = ollama.embeddings(
model='nomic-embed-text', # dedicated embedding model
prompt='Machine learning is a branch of artificial intelligence.'
)
embedding = response['embedding']
print(f"Embedding dimensions: {len(embedding)}")
# Compare two texts by cosine similarity
def cosine_similarity(a, b):
a, b = np.array(a), np.array(b)
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
resp1 = ollama.embeddings(model='nomic-embed-text', prompt='How do transformers work?')
resp2 = ollama.embeddings(model='nomic-embed-text', prompt='Explain the attention mechanism in LLMs.')
resp3 = ollama.embeddings(model='nomic-embed-text', prompt='What is the capital of France?')
sim_12 = cosine_similarity(resp1['embedding'], resp2['embedding'])
sim_13 = cosine_similarity(resp1['embedding'], resp3['embedding'])
print(f"Transformers vs Attention: {sim_12:.3f}") # high similarity
print(f"Transformers vs France: {sim_13:.3f}") # low similarityFor embeddings, use a dedicated embedding model rather than a general-purpose model — nomic-embed-text and mxbai-embed-large are both excellent and fast:
ollama pull nomic-embed-text
ollama pull mxbai-embed-largeTool Calling
Ollama supports structured tool calls for models that have been trained with function calling:
import ollama
import json
tools = [{
'type': 'function',
'function': {
'name': 'get_current_weather',
'description': 'Get the current weather for a given location',
'parameters': {
'type': 'object',
'properties': {
'location': {
'type': 'string',
'description': 'City and country, e.g. "Sydney, AU"'
},
'unit': {
'type': 'string',
'enum': ['celsius', 'fahrenheit']
}
},
'required': ['location']
}
}
}]
response = ollama.chat(
model='llama3.2', # or any tool-capable model
messages=[{'role': 'user', 'content': "What's the weather like in Tokyo right now?"}],
tools=tools
)
if response['message'].get('tool_calls'):
for call in response['message']['tool_calls']:
fn = call['function']
print(f"Model wants to call: {fn['name']}")
print(f"With arguments: {json.dumps(fn['arguments'], indent=2)}")
# Your code would call the actual function here, then feed the result backOpenAI-Compatible Endpoint
Ollama provides an OpenAI-compatible API at /v1/chat/completions. This lets you use the OpenAI Python SDK pointed at your local Ollama — useful for swapping local models into code originally written for OpenAI:
from openai import OpenAI
client = OpenAI(
base_url='http://localhost:11434/v1',
api_key='ollama' # any non-empty string
)
response = client.chat.completions.create(
model='llama3.2',
messages=[
{'role': 'system', 'content': 'You are a helpful assistant.'},
{'role': 'user', 'content': 'Explain Python decorators briefly.'}
]
)
print(response.choices[0].message.content)
# Streaming with OpenAI SDK
stream = client.chat.completions.create(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Count to 10 slowly.'}],
stream=True
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end='', flush=True)This is particularly useful when you’re building apps that need to support both local and cloud models — switch by changing base_url and api_key, no other code changes needed.
Setting Model Options Per-Request
Pass model parameters in the options dict to override defaults for a specific call:
import ollama
response = ollama.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Write me a creative short story opening.'}],
options={
'temperature': 0.9, # higher = more creative
'top_p': 0.95,
'num_ctx': 8192, # context window
'num_predict': 500, # max tokens to generate
'repeat_penalty': 1.1, # reduce repetition
'seed': 42 # reproducible outputs
}
)Key options to know: temperature (0–2, default ~0.8 — lower is more focused, higher is more creative), num_ctx (context window, overrides the model default), num_predict (max output tokens), seed (for reproducible outputs in testing).
Figure 2 — Common API Patterns: Library vs Raw HTTP
Async Usage
For applications where you’re making multiple Ollama calls concurrently or integrating with async frameworks like FastAPI:
import asyncio
import ollama
async def get_summary(text: str, model: str = 'llama3.2') -> str:
client = ollama.AsyncClient()
response = await client.chat(
model=model,
messages=[{
'role': 'user',
'content': f'Summarise this in one sentence: {text}'
}]
)
return response['message']['content']
async def process_batch(texts: list[str]) -> list[str]:
# Process multiple texts concurrently
tasks = [get_summary(text) for text in texts]
return await asyncio.gather(*tasks)
# Run it
texts = [
"The transformer architecture uses self-attention mechanisms...",
"RAG systems combine retrieval with generation...",
"Fine-tuning adapts a pretrained model to specific tasks..."
]
summaries = asyncio.run(process_batch(texts))
for s in summaries:
print(s)Note that Ollama itself processes requests sequentially by default — concurrent async calls will queue unless you’ve set OLLAMA_NUM_PARALLEL higher than 1. For true parallel processing, increase that variable and ensure you have the hardware headroom to support it.
Error Handling
Robust applications need to handle Ollama failures gracefully:
import ollama
from ollama import ResponseError
import time
def chat_with_retry(model: str, messages: list, max_retries: int = 3) -> str:
for attempt in range(max_retries):
try:
response = ollama.chat(model=model, messages=messages)
return response['message']['content']
except ResponseError as e:
if e.status_code == 404:
print(f"Model '{model}' not found. Pulling...")
ollama.pull(model)
continue
print(f"API error (attempt {attempt+1}): {e}")
if attempt < max_retries - 1:
time.sleep(2 ** attempt) # exponential backoff
except Exception as e:
print(f"Unexpected error: {e}")
if attempt < max_retries - 1:
time.sleep(1)
raise RuntimeError(f"Failed after {max_retries} attempts")
result = chat_with_retry('llama3.2', [{'role': 'user', 'content': 'Hello'}])
print(result)The most common errors: 404 (model not downloaded — pull it first), connection errors (Ollama not running — check the service), and timeout errors (model taking too long — increase timeout in the client or reduce request complexity). The retry pattern above handles the common cases cleanly.
Connecting to a Remote Ollama Instance
If Ollama is running on a different machine on your network, pass the host to the client:
import ollama
# Connect to Ollama on another machine
client = ollama.Client(host='http://192.168.1.100:11434')
response = client.chat(
model='llama3.2',
messages=[{'role': 'user', 'content': 'Are you running remotely?'}]
)
print(response['message']['content'])
# With basic auth (if proxy adds it)
import httpx
client_auth = ollama.Client(
host='https://ollama.yourdomain.com',
auth=('username', 'password')
)The Ollama Python library is a thin, well-designed wrapper around the REST API — everything it does can also be done with raw requests calls if you prefer. But for most Python projects, the library saves enough boilerplate on streaming, error handling, and response parsing that it's worth adding as a dependency. The combination of a local model, the Ollama API, and Python gives you a capable, private, cost-free LLM backend for any project you're building.
Building a Simple Chatbot with the Ollama API
Putting together a simple interactive chatbot in the terminal consolidates everything above into one working example:
import ollama
def run_chatbot(model: str = 'llama3.2', system: str = 'You are a helpful assistant.'):
messages = [{'role': 'system', 'content': system}]
print(f"Chatting with {model}. Type 'quit' to exit, 'clear' to reset.
")
while True:
user_input = input("You: ").strip()
if not user_input:
continue
if user_input.lower() == 'quit':
break
if user_input.lower() == 'clear':
messages = [{'role': 'system', 'content': system}]
print("Conversation cleared.
")
continue
messages.append({'role': 'user', 'content': user_input})
print("Assistant: ", end='', flush=True)
full_response = ''
for chunk in ollama.chat(model=model, messages=messages, stream=True):
token = chunk['message']['content']
print(token, end='', flush=True)
full_response += token
print()
messages.append({'role': 'assistant', 'content': full_response})
if __name__ == '__main__':
run_chatbot(model='llama3.2')This pattern — streaming output to the terminal, maintaining message history, and supporting basic commands — is the foundation for almost any interactive LLM application. From here you can add file input (read a document and add it to the first user message), logging (write messages to a file for later review), model switching (let the user change models mid-conversation), or a web interface (wrap the streaming logic in a FastAPI endpoint that sends server-sent events to a browser). The Ollama API's simplicity makes all of these straightforward extensions.
Rate Limiting and Batching Requests
Unlike cloud APIs, Ollama has no rate limits — but it does have a hardware limit: it can only run one request at a time by default (OLLAMA_NUM_PARALLEL=1). If you send multiple concurrent requests, they queue and execute sequentially. For batch processing jobs, this means your throughput is bounded by how fast the model generates on your hardware. The practical optimisation for batch jobs is to send requests sequentially in a tight loop rather than trying to parallelise — queuing overhead from concurrent requests adds latency without improving throughput when the bottleneck is GPU compute. For jobs that need to process thousands of documents, structure your code to pipeline the work: while the model is generating response N, prepare the prompt for request N+1, so you're never waiting on I/O when the model is ready for the next request. The num_predict option is useful for batch jobs where you want shorter, more controlled outputs — setting it to 200 or 300 tokens ensures each request completes quickly and predictably rather than potentially generating several thousand tokens if the model decides to be verbose.
Choosing the Right Approach for Your Project
The ollama Python library is the right default for new projects — it's well-maintained, handles streaming cleanly, and the API surface matches the Ollama REST API closely enough that switching to raw HTTP calls later (if needed) requires minimal changes. Use the OpenAI-compatible endpoint when you need your code to work with both local Ollama and cloud OpenAI models without branching logic — just make the base URL and API key configurable. Use raw requests or httpx calls when you're in an environment where you can't add the ollama package as a dependency, or when you need very fine-grained control over the HTTP behaviour (custom retry logic, proxy configuration, specific header handling). For async applications — FastAPI backends, Discord bots, anything using asyncio — use ollama.AsyncClient(), which gives you the same interface as the sync client but with non-blocking calls that play nicely with Python's async event loop.
What to Build Next
With the Ollama API working in Python, a few directions are particularly worth exploring. The embeddings endpoint opens up semantic search — build a simple vector store with numpy or use ChromaDB locally, embed your documents, and you have a private RAG system running entirely on your machine. The tool calling support makes it straightforward to build agents that can search, calculate, or call your own functions — start with a single tool and a simple ReAct-style loop before adding more complexity. And if you want a web-accessible interface, FastAPI plus the streaming API makes a clean combination: a POST endpoint that streams model responses as server-sent events, callable from any browser client with a few lines of JavaScript. All of these are natural extensions of the patterns in this guide, and the Ollama Python library's consistent interface means the code you've written here transfers directly into any of them without rethinking the fundamentals.