How to Use Ollama with Oobabooga text-generation-webui

by

Oobabooga’s text-generation-webui (often called “ooba”) is one of the most feature-rich local LLM frontends available — a browser-based interface with extensive model loading options, sampling controls, prompt templates, extensions, and API access. It predates many newer tools and has accumulated years of features and community extensions. While Ollama and ooba can both run local LLMs, they approach it differently: Ollama prioritises simplicity and API compatibility, while ooba prioritises maximum control and features. This guide explains how the two tools relate, how to use them together via the API, and when ooba makes more sense than Ollama alone.

Oobabooga vs Ollama: Two Different Tools

It’s worth being clear about how ooba and Ollama differ before discussing integration. Ollama is an inference server — it manages models, exposes an API, and handles the actual LLM computation. Oobabooga’s text-generation-webui is primarily a frontend — it provides a browser interface for interacting with models. Ooba can run models directly (using its own inference backends: transformers, llama.cpp, ExLlamaV2, AutoGPTQ), or it can connect to external inference servers like Ollama via API. The integration pattern covered in this guide uses ooba as a rich frontend with Ollama handling inference in the background — giving you ooba’s feature-rich UI while using Ollama’s model management and inference reliability.

The alternative — running ooba standalone without Ollama — is also valid, particularly if you need ooba’s native loading backends (ExLlamaV2 for GPTQ models, transformers for models not in GGUF format, or specific quantization methods Ollama doesn’t support). This guide focuses on the Ollama-backed path since it’s simpler to set up and benefits from Ollama’s cross-platform support and model management.

Installing text-generation-webui

git clone https://github.com/oobabooga/text-generation-webui.git
cd text-generation-webui
# Linux/Mac:
./start_linux.sh  # or start_macos.sh
# Windows:
start_windows.bat

The startup scripts install a conda environment and dependencies automatically. First run takes 5–10 minutes. Ooba opens at http://localhost:7860 by default.

Connecting Ooba to Ollama via OpenAI API

The cleanest way to use Ollama with ooba is through ooba’s OpenAI extension, which uses the OpenAI-compatible endpoint. In ooba’s interface:

  1. Navigate to the Session tab → Extensions
  2. Enable the openai extension
  3. Restart ooba (--extensions openai flag)
  4. Go to the API tab or Model tab → set the OpenAI API base to http://localhost:11434/v1

Or start ooba with the extension and API URL from the command line:

python server.py --extensions openai --api-port 5000

Then in ooba’s API settings panel, set the base URL to Ollama’s endpoint. Once connected, you can select Ollama models from the dropdown in ooba’s Model tab and interact with them through ooba’s full UI.

What Ooba Adds Over the Ollama CLI

The reason to run ooba as a frontend for Ollama rather than just using Ollama directly comes down to specific ooba features:

Character and persona management. Ooba has a character card system with persistent persona definitions, backstory, example dialogue, and scenario context. For extended roleplay or character-based interactions, this is significantly more organised than manually managing system prompts.

Prompt templates. Ooba includes and correctly applies the chat templates for dozens of models — Llama, Mistral, Alpaca, ChatML, and more. When models have specific prompt formats that affect output quality, ooba applies them correctly. Ollama handles this natively too, but ooba’s template library and customisation options are more extensive.

Notebook mode. A raw text completion interface without the chat wrapper — useful for text generation tasks, fiction writing, or tasks where you want direct control over the full prompt without the chat format applied.

Training interface. Ooba has a LoRA training tab for fine-tuning models on custom datasets directly from the UI. This is independent of Ollama’s inference and uses ooba’s own transformers backend.

Extensions ecosystem. Ooba’s extension system includes gallery generation, web search, speech synthesis, translation, and many community-built additions. These integrate with whatever model is active, including Ollama-backed models via the API.

Figure 1 — Ooba + Ollama vs Standalone Options

Setup Setup ease UI features Sampling control Best for Ollama CLI onlySimplestMinimalVia flags/APIDev, scripting Open WebUI + OllamaEasyGoodLimited UIDaily chat use Ooba + Ollama APIModerateExcellentFull granularPower users Ooba standalone (no Ollama)ComplexExcellentFull granularNon-GGUF models, fine-tuning SillyTavern + OllamaEasyVery goodGoodCharacter/roleplay focus

Calling the Ollama API from Ooba’s Chat Interface

With the OpenAI extension active and pointing at Ollama, the ooba chat interface sends your messages to Ollama for inference and displays the streamed response. The full ooba UI — character cards, prompt templates, generation parameters like temperature and top_p, and extensions — all work with the Ollama backend. You get ooba’s feature richness with Ollama’s model management simplicity.

Some ooba features behave slightly differently when using Ollama via API rather than a natively loaded model. Sampling parameters set in ooba’s interface are passed through the OpenAI-compatible request to Ollama, so they do affect generation. However, ooba’s native loading features (4-bit quantization options, specific ExLlamaV2 settings, GPTQ loading) are only available for models loaded directly by ooba — they don’t apply when Ollama handles inference. If you need those features for a specific model, you’d use ooba in standalone mode rather than the Ollama-backed configuration.

Ooba’s API: Another Option for Developers

Text-generation-webui exposes its own API that some developers target directly rather than using Ollama’s API. If you encounter code that calls the ooba API (http://localhost:5000/api/v1/generate), you can still back it with Ollama by running ooba with the openai extension connected to Ollama — requests to ooba get forwarded to Ollama for inference. This is useful when adapting existing code written for the ooba API:

import requests

# Calling ooba's API (which forwards to Ollama)
response = requests.post(
    "http://localhost:5000/api/v1/chat",
    json={
        "messages": [
            {"role": "user", "content": "Hello from ooba API"}
        ],
        "mode": "chat",
        "character": "Assistant"
    }
)
print(response.json()["choices"][0]["message"]["content"])

Ooba’s own API and Ollama’s API are not interchangeable — they have different schemas — but when ooba is connected to Ollama, the Ollama API remains available at localhost:11434 while ooba’s API operates at localhost:5000. You can use either, or both, depending on which your application expects.

When to Use Ooba vs Other Ollama Frontends

The decision is practical: ooba is the right choice when you specifically need its unique features. Use it for: extensive sampling parameter experimentation (ooba’s UI makes it easy to try many combinations interactively), fine-tuning workflows (ooba’s LoRA training tab works with its own backends), non-GGUF model formats (GPTQ, AWQ, safetensors models not yet converted or available in Ollama’s library), and deep character or persona work (ooba’s character system is more mature than alternatives). For everything else — daily chat, coding assistance, RAG applications, API-backed projects — Ollama with a simpler frontend (Open WebUI, LibreChat, or just the Python library) is less complex and just as capable. Ooba is an excellent tool with a rich feature set, but that richness comes with setup complexity that isn’t necessary for straightforward local AI use. Start with Ollama and a simpler frontend, and reach for ooba when a specific feature it offers becomes the bottleneck in your workflow.

Figure 2 — Ooba Features and Whether Ollama Handles Them Natively

Feature Ooba standalone Ollama alone Ooba + Ollama GGUF model loadingYesYes (better)Ollama handles GPTQ / ExLlamaV2 modelsYesNoOoba handles Sampling param UIExcellentFlags/API onlyOoba UI LoRA trainingYes (transformers)NoOoba handles OpenAI API compatVia extensionBuilt-inOllama handles

The Practical Verdict

Ooba and Ollama are complementary rather than competing tools — ooba was designed for maximum control and flexibility over local LLM inference, Ollama was designed for simplicity and portability. If you’re a power user who regularly needs ooba’s advanced features — particularly its sampling parameter UI, its notebook mode for raw completion, or its extension ecosystem — the ooba + Ollama combination is a strong setup that gives you ooba’s feature richness with Ollama’s model management and cross-platform reliability. If you primarily want a clean, easy-to-use local AI setup and only occasionally need advanced tuning, Ollama alone with a simpler frontend covers your needs without ooba’s additional complexity. Both tools have active development communities and are worth bookmarking for reference even if you don’t use them simultaneously — the Oobabooga community in particular has produced extensive research on sampling strategies and prompting techniques that applies regardless of which inference backend you’re using.

Setting Up Ooba for Long-Term Use

If you decide ooba is the right tool for your workflow, a few setup habits make it more pleasant to maintain. Create a startup script that includes your standard flags so you don’t need to remember them each time — your VRAM size, the extensions you always use, and any API configuration. Pin the ooba version you’re using rather than always running latest — ooba updates can introduce breaking changes to extension APIs or UI layouts, and stability matters more than cutting-edge features for a tool you use daily. Back up your characters, presets, and custom prompt templates — they live in ooba’s data directory and aren’t recreated on reinstall. And document which of ooba’s features you actually use versus which you’ve tried once and never returned to — this helps when you’re evaluating whether to stay on ooba or simplify to a different setup, since ooba’s complexity is only worthwhile if you’re using its unique capabilities. A local LLM setup that’s slightly less powerful but reliably works is more valuable than a maximally configured setup that requires regular troubleshooting.

Leave a Comment