Ruby and Rails applications can integrate Ollama using the ruby-ollama gem or direct HTTP calls via Faraday. This guide covers both approaches with practical examples for Rails APIs and background jobs.
Option 1: ruby-ollama gem
# Gemfile
gem 'ruby-ollama'
bundle installrequire 'ruby_ollama'
client = RubyOllama::Client.new(host: 'http://localhost:11434')
# Chat
response = client.chat(
model: 'llama3.2',
messages: [{ role: 'user', content: 'Why is Ruby popular?' }]
)
puts response.message.content
# Generate
response = client.generate(
model: 'llama3.2',
prompt: 'Write a haiku about Ruby',
stream: false
)
puts response.response
# Embeddings
response = client.embeddings(
model: 'nomic-embed-text',
prompt: 'The quick brown fox'
)
puts response.embedding.length # 768Option 2: Direct HTTP with Faraday
require 'faraday'
require 'json'
class OllamaClient
BASE_URL = 'http://localhost:11434'
def chat(model:, message:, temperature: 0.7)
conn = Faraday.new(url: BASE_URL) do |f|
f.request :json
f.response :json
end
response = conn.post('/api/chat') do |req|
req.body = {
model: model,
messages: [{ role: 'user', content: message }],
stream: false,
options: { temperature: temperature }
}
end
response.body.dig('message', 'content')
end
def embed(model: 'nomic-embed-text', text:)
conn = Faraday.new(url: BASE_URL) do |f|
f.request :json
f.response :json
end
response = conn.post('/api/embeddings') { |r| r.body = { model: model, prompt: text } }
response.body['embedding']
end
end
client = OllamaClient.new
puts client.chat(model: 'llama3.2', message: 'Hello from Ruby!')Rails Service Object
# app/services/ai_service.rb
class AiService
def initialize(model: 'llama3.2')
@model = model
@client = OllamaClient.new
end
def summarise(text, bullet_points: 3)
prompt = "Summarise in #{bullet_points} bullet points:\n\n#{text}"
@client.chat(model: @model, message: prompt, temperature: 0.3)
end
def classify(text, categories:)
prompt = "Classify this text into one of: #{categories.join(', ')}.\nReply with only the category name.\n\n#{text}"
@client.chat(model: @model, message: prompt, temperature: 0.0).strip
end
end
# In a controller:
def create
service = AiService.new
summary = service.summarise(params[:document_text])
render json: { summary: summary }
endSidekiq Background Jobs
# app/jobs/document_summary_job.rb
class DocumentSummaryJob
include Sidekiq::Job
sidekiq_options retry: 3
def perform(document_id)
document = Document.find(document_id)
service = AiService.new
summary = service.summarise(document.content)
document.update!(summary: summary, processed_at: Time.current)
rescue => e
Rails.logger.error "Summary failed for #{document_id}: #{e.message}"
raise
end
end
# Enqueue from controller:
DocumentSummaryJob.perform_async(document.id)Why Ruby for Ollama Integration
Ruby’s expressive syntax and rich ecosystem of web frameworks make it a natural fit for AI-powered web applications. Rails applications can add Ollama-powered features — document summarisation, content classification, intelligent search, response drafting — as service objects and background jobs that integrate naturally with ActiveRecord, ActiveJob, and the Rails request lifecycle. The patterns in this article follow Ruby idioms and Rails conventions, making them easy to adopt for teams already working in the ecosystem.
The ruby-ollama gem provides a clean, idiomatic Ruby API that feels natural alongside other Ruby HTTP clients. For teams that prefer not to add a gem dependency, the Faraday-based approach demonstrates how straightforward the Ollama REST API is to call from any HTTP library — the request and response shapes are simple enough that a custom client is well within reach. Both approaches produce identical results and the choice comes down to team preference for gems versus custom code in the application’s service layer.
Streaming in Rails with ActionController::Live
class AiController < ApplicationController
include ActionController::Live
def stream_response
response.headers['Content-Type'] = 'text/plain; charset=utf-8'
response.headers['Cache-Control'] = 'no-cache'
conn = Faraday.new(url: 'http://localhost:11434')
body = {
model: 'llama3.2',
messages: [{ role: 'user', content: params[:message] }],
stream: true
}.to_json
conn.post('/api/chat', body, 'Content-Type' => 'application/json') do |req|
req.options.on_data = Proc.new do |chunk, _|
data = JSON.parse(chunk) rescue next
content = data.dig('message', 'content')
response.stream.write(content) if content
end
end
ensure
response.stream.close
end
endStructured Output in Ruby
require 'json'
def extract_contact(text, client:)
schema = {
type: 'object',
properties: {
name: { type: 'string' },
email: { type: 'string' },
phone: { type: 'string' }
},
required: %w[name email]
}
conn = Faraday.new(url: 'http://localhost:11434') do |f|
f.request :json
f.response :json
end
response = conn.post('/api/chat') do |req|
req.body = {
model: 'llama3.2',
messages: [{ role: 'user', content: "Extract contact from: #{text}" }],
format: schema,
stream: false,
options: { temperature: 0 }
}
end
JSON.parse(response.body.dig('message', 'content'))
end
contact = extract_contact('Call Alice at alice@example.com or 555-1234', client: nil)
puts contact['name'] # 'Alice'
puts contact['email'] # 'alice@example.com'Testing Ruby Ollama Code
# spec/services/ai_service_spec.rb
RSpec.describe AiService do
let(:service) { described_class.new }
describe '#summarise' do
it 'returns a summary string' do
# Use VCR or WebMock to stub Ollama HTTP calls in tests
stub_request(:post, 'http://localhost:11434/api/chat')
.to_return(
status: 200,
body: { message: { content: 'Summary: Key points here.' }, done: true }.to_json,
headers: { 'Content-Type' => 'application/json' }
)
result = service.summarise('Long document text...')
expect(result).to include('Summary')
end
end
endGetting Started
Add the Faraday gem (already in most Rails apps) or ruby-ollama to your Gemfile, create an OllamaClient service class, and use it in a controller action or ActiveJob. The background job pattern with Sidekiq is the most practical approach for document processing and batch operations — it follows the same conventions as any other Sidekiq job in a Rails application and benefits from Sidekiq’s retry logic, monitoring UI, and concurrency management. Start with the synchronous service object for interactive features and add the Sidekiq job when you need async processing for longer-running inference tasks.
Ollama’s OpenAI-compatible REST API works with any Ruby HTTP client. The ruby-openai gem (pointed at Ollama’s /v1 endpoint) is the most convenient option, and the community ruby-ollama gem provides Ollama-native API access. This guide covers both approaches for integrating Ollama into Ruby and Rails applications.
Option 1: ruby-openai Gem (Recommended)
gem install ruby-openai
# or in Gemfile: gem 'ruby-openai'require 'openai'
client = OpenAI::Client.new(
access_token: 'ollama', # any non-empty string
uri_base: 'http://localhost:11434/v1'
)
# Chat
response = client.chat(
parameters: {
model: 'llama3.2',
messages: [{ role: 'user', content: 'Why is Ruby elegant?' }]
}
)
puts response.dig('choices', 0, 'message', 'content')
# Streaming
client.chat(
parameters: {
model: 'llama3.2',
messages: [{ role: 'user', content: 'Count to 5.' }],
stream: proc do |chunk, _bytesize|
print chunk.dig('choices', 0, 'delta', 'content')
end
}
)Embeddings
response = client.embeddings(
parameters: {
model: 'nomic-embed-text',
input: 'The quick brown fox'
}
)
vector = response['data'][0]['embedding']
puts "Dimensions: #{vector.length}"Rails Service Object
# app/services/ai_service.rb
class AiService
def initialize
@client = OpenAI::Client.new(
access_token: 'ollama',
uri_base: ENV.fetch('OLLAMA_HOST', 'http://localhost:11434') + '/v1'
)
end
def summarise(text, model: 'llama3.2')
response = @client.chat(
parameters: {
model: model,
messages: [{
role: 'user',
content: "Summarise in 3 bullet points:\n\n#{text}"
}],
temperature: 0.3
}
)
response.dig('choices', 0, 'message', 'content')
end
def classify(text)
response = @client.chat(
parameters: {
model: 'llama3.2',
messages: [{
role: 'user',
content: "Classify as: invoice, contract, email, or other. Reply with JSON only.\n\n#{text[0..1000]}"
}],
temperature: 0
}
)
JSON.parse(response.dig('choices', 0, 'message', 'content'))
end
end
# Usage in a controller
class DocumentsController < ApplicationController
def analyse
ai = AiService.new
@summary = ai.summarise(params[:text])
@classification = ai.classify(params[:text])
render json: { summary: @summary, classification: @classification }
end
endBackground Jobs with ActiveJob
# app/jobs/summarise_document_job.rb
class SummariseDocumentJob < ApplicationJob
queue_as :default
def perform(document_id)
document = Document.find(document_id)
ai = AiService.new
summary = ai.summarise(document.content)
document.update!(summary: summary, summarised_at: Time.current)
end
end
# Enqueue from controller
SummariseDocumentJob.perform_later(document.id)Rails Credentials for Configuration
# config/credentials.yml.enc (edit with: rails credentials:edit)
# ollama:
# host: http://localhost:11434
# Access in code:
ollama_host = Rails.application.credentials.dig(:ollama, :host)Why Ruby for AI Integration
Ruby's expressive syntax and the Rails ecosystem's conventions make AI integration feel natural — service objects, background jobs, and controller concerns are the established patterns for encapsulating external service calls, and Ollama fits cleanly into all three. The ruby-openai gem's OpenAI-compatible interface means any team already using OpenAI in a Rails application can switch to Ollama by changing two lines: the access_token and uri_base initialisation arguments. This drop-in replacement story is one of the most compelling aspects of Ollama's OpenAI-compatible API for Ruby teams — no new gem, no new API patterns, just updated configuration.
Streaming in Rails with ActionController::Live
class AiController < ApplicationController
include ActionController::Live
def stream
response.headers['Content-Type'] = 'text/event-stream'
response.headers['Cache-Control'] = 'no-cache'
client = OpenAI::Client.new(access_token: 'ollama',
uri_base: 'http://localhost:11434/v1')
client.chat(
parameters: {
model: 'llama3.2',
messages: [{ role: 'user', content: params[:message] }],
stream: proc do |chunk, _bytesize|
token = chunk.dig('choices', 0, 'delta', 'content')
response.stream.write("data: #{token.to_json}\n\n") if token
end
}
)
ensure
response.stream.close
end
endCaching AI Responses
class AiService
def summarise_with_cache(text, model: 'llama3.2')
cache_key = "ai/summary/#{Digest::SHA256.hexdigest(text + model)}"
Rails.cache.fetch(cache_key, expires_in: 1.day) do
summarise(text, model: model) # Only calls Ollama on cache miss
end
end
endTesting Rails AI Services
# spec/services/ai_service_spec.rb
RSpec.describe AiService do
describe '#summarise' do
it 'returns a summary string' do
# Stub the OpenAI client to avoid real Ollama calls in tests
stub_response = {
'choices' => [{ 'message' => { 'content' => '• Point 1\n• Point 2\n• Point 3' } }]
}
allow_any_instance_of(OpenAI::Client).to receive(:chat).and_return(stub_response)
service = AiService.new
result = service.summarise('Some document text')
expect(result).to include('Point 1')
end
end
endGetting Started
Add gem 'ruby-openai' to your Gemfile, configure the client to point at http://localhost:11434/v1 with access_token: 'ollama', and call client.chat with your model and messages. The AiService service object pattern from this article provides a clean encapsulation layer that makes swapping models, adding caching, and writing tests straightforward. For Rails applications already using OpenAI, the migration to Ollama is a configuration change — the rest of your application code, controllers, jobs, and tests remain unchanged. Pull your chosen model with ollama pull llama3.2, update the uri_base, and your AI features are running locally with no API costs and no data leaving your infrastructure.
The ruby-openai Gem vs Ruby-Native Ollama Gems
Two main options exist for Ollama integration in Ruby: the ruby-openai gem (pointing at Ollama's OpenAI-compatible /v1 endpoint) and purpose-built gems like ruby-ollama or ollama-ai that wrap the native Ollama API. The ruby-openai approach is recommended for most Rails teams for one key reason: it is the same gem you would use if you switched to OpenAI, Anthropic's API, or another OpenAI-compatible provider. The interface is identical regardless of backend — switching from Ollama to OpenAI is a two-line configuration change, and vice versa. Purpose-built Ollama gems have access to Ollama-native features (model management, Modelfile creation, streaming with the native format) but at the cost of a non-portable interface that requires changes if you ever switch backends. For typical Rails AI features — chat, summarisation, classification, embeddings — the OpenAI-compatible interface is fully sufficient.
Handling Long-Running Requests with Rack Timeout
Rails applications typically configure Rack::Timeout to kill requests that exceed a certain duration (commonly 15–30 seconds). LLM inference can easily exceed this limit for long responses. Configure Rack::Timeout to allow longer timeouts for AI endpoints specifically, or use the background job pattern (ActiveJob) to move inference out of the request-response cycle entirely:
# config/initializers/rack_timeout.rb
# Increase global timeout for AI-heavy applications
Rack::Timeout.timeout = 120 # 2 minutes
# Or use per-request timeout via middleware:
Rack::Timeout::StateManager.timeout_for_request do |env|
env['PATH_INFO'].start_with?('/ai') ? 300 : 15
endEnvironment-Based Configuration
# config/initializers/ai.rb
AI_CLIENT = OpenAI::Client.new(
access_token: ENV.fetch('OLLAMA_API_KEY', 'ollama'),
uri_base: ENV.fetch('OLLAMA_HOST', 'http://localhost:11434') + '/v1',
request_timeout: ENV.fetch('OLLAMA_TIMEOUT', '120').to_i
)
# config/environments/production.rb — point at team server
# OLLAMA_HOST=http://ai-server.internal:11434
# config/environments/test.rb — stub in tests
# (no real Ollama calls in test environment)Ruby AI Beyond Rails
The ruby-openai gem works in any Ruby context — Sinatra, Rack apps, plain Ruby scripts, and Jekyll plugins. For non-Rails Ruby projects, the same client initialisation and chat/embedding API calls work identically. A Ruby script for batch document processing is a common simple use case: read files, call ollama chat for each, write results — no framework needed, just the gem and a few dozen lines of Ruby. The simplicity of the HTTP API combined with Ruby's clean syntax makes this kind of one-off AI processing script one of the fastest things to write in the language.
Performance Considerations for Ruby
Ruby's GIL (Global Interpreter Lock) means MRI Ruby cannot run truly parallel threads, but this does not significantly affect Ollama integration because the bottleneck is always the inference step, not the Ruby code. For concurrent AI requests in Rails, use Puma with multiple workers (processes, not threads) and ensure each Puma worker has its own AI client instance. For high-throughput batch processing outside Rails, use the Parallel gem or plain Ruby processes (via Process.fork) to process documents in parallel, with each process making its own sequential Ollama calls. Jruby (which lacks the GIL) enables true thread-based parallelism for I/O-bound tasks, though the Ollama calls are I/O-bound and benefit from concurrent threading even in MRI Ruby via Puma's threaded mode.
The Ruby AI Ecosystem in 2026
Ruby's AI integration ecosystem has matured considerably. The ruby-openai gem is actively maintained and covers the full OpenAI API including assistants, fine-tuning, and moderation in addition to chat and embeddings. Community gems for LangChain-style orchestration in Ruby (langchainrb) provide chains, agents, and vector store integrations following patterns familiar from the Python ecosystem. For Rails teams building AI features, the combination of ruby-openai for model calls, ActiveJob for async processing, Rails.cache for response caching, and LangChainrb for complex orchestration covers the vast majority of AI application requirements without leaving the Ruby ecosystem.
Ollama's OpenAI-compatible API makes Ruby the language of least resistance for teams already invested in Rails. No new patterns to learn, no separate service to deploy, no context-switching to Python for AI features — just a new gem pointed at a local endpoint. For Rails shops that have been waiting for the right moment to add AI capabilities to their applications, the combination of Ollama's local inference and the mature Ruby AI gem ecosystem makes 2026 that moment.
Migrating from OpenAI to Ollama in Rails
For Rails applications already calling the OpenAI API via ruby-openai, migrating to Ollama is straightforward in most cases. Update the client initialisation to point at your Ollama endpoint, change the model names (from gpt-4o to llama3.2 or your preferred model), and test. Chat and embedding calls use identical method signatures — the response structure is the same OpenAI-compatible JSON. The areas that need attention: Ollama's model names do not map one-to-one with OpenAI model names, so prompts optimised for GPT-4o may need adjustment for a different model family; tool calling (function calling) support varies by model, so verify your chosen model supports it if you use tools; and image inputs require a vision-capable model like LLaVA or Gemma 3 rather than any model. Beyond these three areas, the migration is typically a configuration change, not a code change — which is the central practical value of Ollama's OpenAI-compatible API for Ruby teams.
Practical Starting Point
Add the gem, configure the client pointing at http://localhost:11434/v1, write the AiService class from this article, and call it from one controller action or background job. Evaluate the output quality on your actual use case with the model you have chosen — run 10–20 real examples through it and compare to your quality bar. If the quality is acceptable, expand the integration; if not, try a larger or different model before spending more time on the integration. The cost of evaluating local Ollama for your Rails application is under an hour, and the potential savings in API costs and privacy improvements make that hour one of the best-spent in any AI-integration project.
A Note on Model Quality in Ruby Applications
The model you choose matters more than the integration library. A well-prompted llama3.2 7B handles most common Rails AI tasks — document classification, summarisation, simple Q&A — with quality that is good enough for production use in internal tools and non-critical customer features. For customer-facing features where quality is paramount, evaluate larger models (13B, 32B) or cloud models alongside the local option before committing to a production architecture. The ruby-openai gem makes this evaluation easy: switching between Ollama's llama3.2, a locally-hosted Qwen 32B, and OpenAI's GPT-4o requires only changing the model name and uri_base — your service object code is identical across all three. Run the same 20–30 representative examples through each option and let your quality requirements drive the choice rather than defaulting to either local or cloud — the evaluation cost is an hour, and the data you get from running real examples is far more informative than any benchmark or comparison article for your specific use case and your production requirements far better than any general comparison can — and the investment in evaluation is almost always returned within the first week of production usage through either cost savings, privacy improvements, or both — a straightforward return on a minimal upfront investment that every Ruby team working with AI should make before committing to any architecture decision about their AI backend.
The model-agnostic interface that ruby-openai provides is ultimately what makes local AI integration in Ruby so low-friction — you evaluate on Ollama, and if you ever need to switch to a cloud model for quality reasons, the gem handles it without touching your application code.
That shift from prototype to production without touching application code is the most practical advantage the library abstraction provides — and it compounds every time you add a new feature or scale a new model to a new use case.