Streamlit and Gradio are Python libraries that let you build web interfaces for AI applications in a few dozen lines of code. Both are excellent choices for wrapping an Ollama model in a shareable, polished UI — without writing HTML, CSS, or JavaScript. Whether you’re building a demo to share with colleagues, a personal tool for a specific workflow, or a production-ready internal application, one of these two libraries is usually the fastest path from “Ollama model” to “proper web app.” This guide covers how to build streaming chat apps with both, what each is best suited for, and how to deploy them.
Streamlit: Pythonic App Builder
Streamlit turns Python scripts into interactive web apps. You write Python, Streamlit renders a UI. It’s particularly good for data-heavy applications — if your Ollama app needs charts, tables, file uploads, or sidebar controls alongside the chat interface, Streamlit handles all of it naturally. Install it:
pip install streamlit ollamaA Streaming Chat App in Streamlit
import streamlit as st
import ollama
st.title("Local AI Chat")
st.caption("Powered by Ollama")
# Model selector in sidebar
with st.sidebar:
model = st.selectbox(
"Model",
[m.model for m in ollama.list().models],
index=0
)
if st.button("Clear conversation"):
st.session_state.messages = []
st.rerun()
# Initialise conversation history
if "messages" not in st.session_state:
st.session_state.messages = []
# Display existing messages
for msg in st.session_state.messages:
with st.chat_message(msg["role"]):
st.markdown(msg["content"])
# Handle new input
if prompt := st.chat_input("Message..."):
st.session_state.messages.append({"role": "user", "content": prompt})
with st.chat_message("user"):
st.markdown(prompt)
with st.chat_message("assistant"):
response_placeholder = st.empty()
full_response = ""
# Stream the response
stream = ollama.chat(
model=model,
messages=st.session_state.messages,
stream=True
)
for chunk in stream:
full_response += chunk["message"]["content"]
response_placeholder.markdown(full_response + "▌")
response_placeholder.markdown(full_response)
st.session_state.messages.append({
"role": "assistant",
"content": full_response
})Run with: streamlit run app.py. This gives you a full streaming chat interface with model selection, conversation history, and a clear button — in about 50 lines of Python. The cursor character (▌) while streaming shows the model is generating, which makes the experience feel responsive.
Adding a System Prompt Control
with st.sidebar:
system_prompt = st.text_area(
"System prompt",
value="You are a helpful assistant. Be concise.",
height=100
)
# Prepend system message to every API call
messages_with_system = [
{"role": "system", "content": system_prompt}
] + st.session_state.messagesAdd this to the sidebar and use messages_with_system in the ollama.chat() call. Now the system prompt is editable from the UI without changing code — useful for apps where you want to quickly test different prompt approaches.
Gradio: The ML Demo Standard
Gradio is the go-to library for ML demos — it’s what most Hugging Face Spaces use, and it’s purpose-built for sharing AI applications quickly. Gradio’s interface components (ChatInterface, Image, Audio, DataFrame) are designed for AI use cases, and the sharing mechanism is unbeatable: one argument adds a public shareable link via Gradio’s servers.
pip install gradio ollamaA Streaming Chat App in Gradio
import gradio as gr
import ollama
def chat(message, history, model, system_prompt):
messages = []
if system_prompt:
messages.append({"role": "system", "content": system_prompt})
# Convert Gradio history format to Ollama format
for user_msg, assistant_msg in history:
messages.append({"role": "user", "content": user_msg})
messages.append({"role": "assistant", "content": assistant_msg})
messages.append({"role": "user", "content": message})
# Stream the response
response = ""
for chunk in ollama.chat(model=model, messages=messages, stream=True):
response += chunk["message"]["content"]
yield response
available_models = [m.model for m in ollama.list().models]
with gr.Blocks(title="Local AI Chat") as demo:
gr.Markdown("# Local AI Chat — Powered by Ollama")
with gr.Row():
model_dropdown = gr.Dropdown(
choices=available_models,
value=available_models[0] if available_models else None,
label="Model"
)
system_input = gr.Textbox(
value="You are a helpful assistant.",
label="System prompt",
lines=2
)
chat_interface = gr.ChatInterface(
fn=chat,
additional_inputs=[model_dropdown, system_input],
title=""
)
demo.launch()Run with: python app.py. Open http://localhost:7860. To get a public shareable link (useful for demos): demo.launch(share=True) — Gradio creates a temporary public URL that forwards to your local app.
Figure 1 — Streamlit vs Gradio: When to Use Each
Building a Document Q&A App with Streamlit
Beyond simple chat, Streamlit’s file upload and session state make it well-suited for document Q&A apps. Here’s a pattern that lets users upload a file and ask questions about it:
import streamlit as st
import ollama
st.title("Document Q&A")
uploaded_file = st.file_uploader("Upload a document", type=["txt", "md"])
if uploaded_file:
doc_text = uploaded_file.read().decode("utf-8")
st.success(f"Loaded: {uploaded_file.name} ({len(doc_text)} chars)")
if "doc_messages" not in st.session_state:
st.session_state.doc_messages = []
for msg in st.session_state.doc_messages:
with st.chat_message(msg["role"]):
st.markdown(msg["content"])
if question := st.chat_input("Ask about the document..."):
st.session_state.doc_messages.append({"role": "user", "content": question})
with st.chat_message("user"):
st.markdown(question)
messages = [
{"role": "system", "content": f"Answer questions about this document:
{doc_text[:8000]}"},
] + st.session_state.doc_messages
with st.chat_message("assistant"):
placeholder = st.empty()
full = ""
for chunk in ollama.chat(model="llama3.2", messages=messages, stream=True):
full += chunk["message"]["content"]
placeholder.markdown(full + "▌")
placeholder.markdown(full)
st.session_state.doc_messages.append({"role": "assistant", "content": full})This is a lightweight RAG pattern without a vector database — for documents under ~6,000 words, stuffing the entire document into the context window is simpler and often more accurate than chunked retrieval. For larger documents, use LlamaIndex or ChromaDB for proper vector-based retrieval.
Image Analysis with Gradio and LLaVA
import gradio as gr
import ollama
import base64
from pathlib import Path
def analyse_image(image_path, question):
if not image_path:
return "Please upload an image."
img_data = base64.b64encode(Path(image_path).read_bytes()).decode()
response = ollama.chat(
model="llava",
messages=[{
"role": "user",
"content": question or "Describe this image in detail.",
"images": [img_data]
}]
)
return response["message"]["content"]
demo = gr.Interface(
fn=analyse_image,
inputs=[
gr.Image(type="filepath", label="Upload image"),
gr.Textbox(label="Question (optional)", placeholder="What do you see?")
],
outputs=gr.Textbox(label="Analysis", lines=8),
title="Local Image Analysis — LLaVA via Ollama",
description="Upload an image and ask questions about it. Runs locally."
)
demo.launch()Gradio’s gr.Image component handles the file upload and temporary path management. The vision model call is standard Ollama API — pass the base64-encoded image in the images list. This entire vision app is about 25 lines of Python.
Deployment Options
Local only: The default. Run the script, access at localhost. Good for personal tools and development.
Local network sharing: streamlit run app.py --server.address 0.0.0.0 or demo.launch(server_name="0.0.0.0") — makes the app accessible to other devices on your local network. Useful for sharing with colleagues on the same WiFi without deploying anywhere.
Gradio public link: demo.launch(share=True) creates a temporary public URL (72-hour lifetime) via Gradio’s tunnel servers. The fastest way to share a demo with someone not on your network. Note: your local Ollama instance handles inference, so the app only works while your machine is running.
Streamlit Cloud: Free hosting for public Streamlit apps at streamlit.io/cloud. The catch: your Ollama instance needs to be accessible from the internet (a VPS or tunnelled connection) since Streamlit Cloud runs the Python code remotely. This works for demo apps where you deploy Ollama on a cloud VM.
Docker container: Containerise your Streamlit or Gradio app alongside Ollama using a multi-container setup. Production deployment for internal tools. Requires more configuration but gives you a fully portable, self-contained AI application.
Figure 2 — App Types and Recommended Framework
Choosing Between Streamlit and Gradio
For most Ollama-backed apps, either framework works well and the choice comes down to what you need beyond the basic chat interface. If your app needs charts, tables, data visualisation, or a complex sidebar layout with multiple controls, Streamlit handles these more naturally — it has better support for data-heavy UIs and the layout system is more flexible. If you primarily need a clean input-output interface for a model (text in, text out; image in, text out), Gradio’s components are more purpose-built for this and the code is more concise. Gradio’s instant sharing is also a significant practical advantage when you want to show something to someone quickly without any deployment overhead. For internal tools you’ll maintain long-term, Streamlit tends to be easier to structure as your app grows in complexity. For demos and prototypes you want to share quickly, Gradio is usually faster to get to a shareable state. Both are production-ready Python libraries with active development and large user bases — either choice is a solid foundation for building Ollama-backed web applications.
Performance and State Management in Streamlit
Streamlit reruns the entire script on every user interaction — which is its core design, but requires careful state management for chat apps. The st.session_state dictionary persists across reruns, which is how conversation history survives each message. Understanding this rerun model prevents common bugs: don’t put expensive operations (model loading, large file reads) at the top level of your script, because they’ll re-execute on every message. Instead, use @st.cache_resource for resources that should be loaded once per session, and @st.cache_data for functions that return data that should be cached between identical calls. For the Ollama client itself, the ollama library maintains an HTTP client internally — you don’t need to cache it. But if you’re loading embedding models, building vector indexes, or reading large configuration files as part of your app setup, cache those with the appropriate decorator. Streamlit’s caching makes these operations feel instant after the first run, which is important for apps that start up frequently or where users navigate between pages that share expensive resources.
Error Handling and User Experience
Production-quality Streamlit and Gradio apps need error handling around Ollama calls — the model might be loading, the service might be temporarily unavailable, or a very long generation might hit a timeout. Wrap your Ollama calls in try-except blocks and surface errors clearly in the UI rather than letting exceptions bubble up to the user as a cryptic traceback. In Streamlit, st.error("Something went wrong: " + str(e)) displays a red error box. In Gradio, raising a gr.Error exception shows a clean error modal. Both are significantly better UX than the raw Python exception that appears without error handling. For streaming specifically, handle the case where the stream ends unexpectedly — a generator that raises mid-stream should be caught and the partial response preserved rather than discarded. Also consider adding a timeout to your Ollama calls for apps where users might submit very long prompts to slow models — a 60-second timeout prevents the interface from appearing permanently frozen on a slow generation. These small UX investments make the difference between an app that feels prototype-quality and one that feels reliable enough to share or deploy seriously.
From Prototype to Internal Tool
The typical path for Streamlit and Gradio Ollama apps: start with the minimal working example (the chat app above), add the specific features your use case needs (file upload, system prompt control, model switching), then harden it for regular use (error handling, sensible defaults, helpful placeholder text, a clear title explaining what the app does). At that point you have something worth deploying beyond your own machine. For internal tools that 5–20 people will use, a simple Docker deployment on a machine with Ollama installed is the most practical path — one container for the Streamlit or Gradio app, Ollama running natively on the host, and a reverse proxy for HTTPS. For demos that need to be shareable without infrastructure, Gradio’s share=True handles it immediately at the cost of the 72-hour link lifetime. The combination of Python’s rapid development speed, Streamlit or Gradio’s zero-HTML UI building, and Ollama’s local inference means you can go from idea to working shared app in a few hours — which is the genuine value proposition of this combination for developers who want to build AI tools without becoming full-stack web developers.
Multi-Model Apps: Switching Models at Runtime
One of the most useful features you can add to a Streamlit or Gradio Ollama app is runtime model switching — letting users pick between your downloaded models without restarting the app. The Ollama Python library makes this trivial since model selection is just a parameter in each API call rather than a client-level configuration. The pattern is identical in both frameworks: fetch the model list once on startup using ollama.list(), populate a dropdown with the results, and pass the selected model name into each ollama.chat() call. Ollama handles the model loading and unloading transparently — if a user switches from Llama 3.2 to Qwen 2.5 Coder, Ollama unloads the first model and loads the second the first time a request is made with the new model (within the keep-alive window). The only UX consideration is the loading delay when switching to a model that isn’t currently loaded in memory — worth noting in your UI with a brief “model may take a moment to load” message next to the selector, so users understand why the first response after a model switch is slower than subsequent ones.
Real-World Use Cases That Work Well
A few Streamlit and Gradio + Ollama applications that solve real problems and are practical to build in an afternoon. A meeting transcript analyser: upload a transcript text file, the app extracts action items, decisions, and open questions into a structured format using a local model, and lets you ask follow-up questions about the meeting content. A code review assistant: paste code into a text area, the app runs it through Qwen 2.5 Coder with a code review prompt and returns categorised feedback (bugs, style issues, security concerns, improvement suggestions) in a structured layout. A batch email drafter: upload a CSV of contacts with context fields, the app generates personalised email drafts for each row using an Ollama model and exports the results back to CSV — a workflow that would otherwise involve expensive API calls or manual writing. A local knowledge base search: index a directory of markdown notes or documentation files at startup, embed them with nomic-embed-text via Ollama, and build a semantic search interface that retrieves and summarises relevant sections when you type a question. Each of these takes 50–150 lines of Python with Streamlit or Gradio, runs entirely locally, and produces a polished UI that non-technical users can interact with comfortably.