ChromaDB is the right starting point for local RAG development, but production systems handling significant query volume, large document collections, or multi-user deployments often need more. Weaviate and Pinecone are two of the most capable production vector databases, each with a distinct positioning. This guide covers concrete setup and usage for both, with code you can run today.
Weaviate for Production RAG
Weaviate’s defining features for production RAG: native hybrid search (BM25 keyword + vector in a single query), built-in multi-tenancy for user-isolated collections, a powerful schema system, and support for multiple vector spaces per object. The easiest way to run Weaviate is Docker locally or Weaviate Cloud for managed production:
# Local Docker (development)
docker run -d -p 8080:8080 -p 50051:50051 \
-e QUERY_DEFAULTS_LIMIT=25 \
-e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \
cr.weaviate.io/semitechnologies/weaviate:latest
# Install Python client
pip install weaviate-clientimport weaviate
from weaviate.classes.init import Auth
from weaviate.classes.config import Configure, Property, DataType
from weaviate.classes.query import MetadataQuery, Filter
# Connect to local Weaviate
client = weaviate.connect_to_local()
# Or Weaviate Cloud
# client = weaviate.connect_to_weaviate_cloud(
# cluster_url="YOUR_CLUSTER_URL",
# auth_credentials=Auth.api_key("YOUR_API_KEY")
# )
# Create a collection with hybrid search enabled
docs = client.collections.create(
name="Documents",
vectorizer_config=Configure.Vectorizer.none(), # we supply vectors ourselves
properties=[
Property(name="content", data_type=DataType.TEXT),
Property(name="source", data_type=DataType.TEXT),
Property(name="chunk", data_type=DataType.INT),
]
)Indexing Documents into Weaviate
import ollama
def embed(text: str) -> list[float]:
return ollama.embeddings(model="nomic-embed-text", prompt=text)["embedding"]
# Batch insert for efficiency
with docs.batch.dynamic() as batch:
for i, chunk_text in enumerate(chunks):
batch.add_object(
properties={
"content": chunk_text,
"source": "handbook.pdf",
"chunk": i
},
vector=embed(chunk_text)
)
print(f"Indexed {len(chunks)} chunks")
client.close()Hybrid Search in Weaviate
Hybrid search is Weaviate’s strongest RAG advantage over ChromaDB. A single query combines BM25 keyword matching and vector similarity, with configurable weighting between them. The alpha parameter controls the blend: 0.0 is pure BM25, 1.0 is pure vector, 0.5 is equal weight:
client = weaviate.connect_to_local()
docs = client.collections.get("Documents")
def hybrid_retrieve(query: str, n: int = 5, alpha: float = 0.7) -> list[str]:
"""Hybrid search: alpha=0.7 weights vector similarity more than BM25."""
query_vector = embed(query)
results = docs.query.hybrid(
query=query, # for BM25 component
vector=query_vector, # for vector component
alpha=alpha,
limit=n,
return_metadata=MetadataQuery(score=True)
)
return [obj.properties["content"] for obj in results.objects]
# Pure vector search
def vector_retrieve(query: str, n: int = 5) -> list[str]:
results = docs.query.near_vector(
near_vector=embed(query),
limit=n,
filters=Filter.by_property("source").equal("handbook.pdf")
)
return [obj.properties["content"] for obj in results.objects]
# Use in RAG pipeline
chunks = hybrid_retrieve("remote work policy", alpha=0.5)
context = "\n\n---\n\n".join(chunks)
client.close()Weaviate Multi-Tenancy
Multi-tenancy is Weaviate’s standout feature for SaaS RAG applications where different users or organisations should see only their own documents. Each tenant gets an isolated partition of the collection with separate storage:
from weaviate.classes.config import Configure
from weaviate.classes.tenants import Tenant
# Create a multi-tenant collection
client.collections.create(
name="UserDocs",
multi_tenancy_config=Configure.multi_tenancy(enabled=True),
properties=[
Property(name="content", data_type=DataType.TEXT),
Property(name="source", data_type=DataType.TEXT),
]
)
collection = client.collections.get("UserDocs")
# Add tenants
collection.tenants.create([Tenant(name="user_alice"), Tenant(name="user_bob")])
# Insert into a specific tenant
alice_col = collection.with_tenant("user_alice")
with alice_col.batch.dynamic() as batch:
batch.add_object(
properties={"content": "Alice's document", "source": "alice_files"},
vector=embed("Alice's document")
)
# Query is automatically scoped to that tenant
alice_results = alice_col.query.near_vector(
near_vector=embed("show my documents"),
limit=5
)
# Bob cannot see Alice's documents and vice versaMulti-tenancy with per-tenant isolation is complex to implement with ChromaDB (separate collections per user) or Qdrant (namespace filtering). Weaviate’s first-class multi-tenancy makes it the natural choice for any RAG product serving multiple users or organisations from a shared infrastructure.
Figure 1 — Weaviate vs Pinecone: Production RAG Comparison
Pinecone for Production RAG
Pinecone is a fully managed vector database with no self-hosted option. Its value proposition is operational simplicity: create an index via API, insert vectors, query — no infrastructure to manage. Pinecone supports hybrid search via sparse-dense retrieval, where sparse vectors (BM25 or SPLADE keyword weights) are combined with dense embedding vectors. The free Starter plan provides one index with up to 100K vectors, sufficient for prototypes and small production deployments. Production plans are usage-based — priced per vector stored and per query, which is competitive with managed Weaviate at small scale but accumulates at large scale compared to self-hosted alternatives.
pip install pinecone-client
from pinecone import Pinecone, ServerlessSpec
pc = Pinecone(api_key="YOUR_API_KEY")
# Create an index
pc.create_index(
name="rag-documents",
dimension=768, # matches nomic-embed-text output dimension
metric="cosine",
spec=ServerlessSpec(cloud="aws", region="us-east-1")
)
index = pc.Index("rag-documents")Indexing and Querying with Pinecone
import ollama
def embed(text: str) -> list[float]:
return ollama.embeddings(model="nomic-embed-text", prompt=text)["embedding"]
# Upsert vectors in batches (Pinecone recommends batches of 100)
def index_chunks(chunks: list[str], source: str):
batch_size = 100
for i in range(0, len(chunks), batch_size):
batch = chunks[i:i+batch_size]
vectors = [
{
"id": f"{source}_{i+j}",
"values": embed(chunk),
"metadata": {"text": chunk, "source": source, "chunk_idx": i+j}
}
for j, chunk in enumerate(batch)
]
index.upsert(vectors=vectors)
print(f"Indexed {len(chunks)} chunks from {source}")
# Query
def retrieve(query: str, n: int = 5, source_filter: str = None) -> list[str]:
filter_dict = {"source": {"$eq": source_filter}} if source_filter else None
results = index.query(
vector=embed(query),
top_k=n,
include_metadata=True,
filter=filter_dict
)
return [match["metadata"]["text"] for match in results["matches"]]
# Index and query
index_chunks(my_chunks, "handbook")
results = retrieve("remote work policy", n=5, source_filter="handbook")Pinecone Namespaces for Multi-Tenancy
# Use namespaces to isolate different users or organisations
def index_for_user(chunks: list[str], source: str, user_id: str):
vectors = [
{"id": f"{user_id}_{source}_{i}",
"values": embed(chunk),
"metadata": {"text": chunk, "source": source}}
for i, chunk in enumerate(chunks)
]
# namespace isolates data per user
index.upsert(vectors=vectors, namespace=user_id)
def retrieve_for_user(query: str, user_id: str, n: int = 5) -> list[str]:
results = index.query(
vector=embed(query),
top_k=n,
include_metadata=True,
namespace=user_id # only searches this user's vectors
)
return [m["metadata"]["text"] for m in results["matches"]]Pinecone namespaces provide logical isolation — vectors in different namespaces cannot be retrieved by each other’s queries. This is simpler than Weaviate’s multi-tenancy (one index with namespace tags vs. first-class tenant management), but sufficient for most multi-user RAG applications. Namespace billing is shared across the index, so all tenants’ usage contributes to the same usage-based cost.
Pinecone Hybrid Search
from pinecone_text.sparse import BM25Encoder
# Fit BM25 on your corpus first
bm25 = BM25Encoder()
bm25.fit(all_chunk_texts)
# Upsert with both dense and sparse vectors
vectors = [{
"id": f"doc_{i}",
"values": embed(chunk), # dense vector
"sparse_values": bm25.encode_documents([chunk])[0], # sparse BM25 vector
"metadata": {"text": chunk}
} for i, chunk in enumerate(chunks)]
index.upsert(vectors=vectors)
# Hybrid query (alpha controls dense/sparse balance)
results = index.query(
vector=embed(query),
sparse_vector=bm25.encode_queries([query])[0],
top_k=5,
alpha=0.7, # 0=sparse only, 1=dense only
include_metadata=True
)Pinecone’s hybrid search requires the pinecone-text package for BM25 encoding. The sparse-dense combination consistently outperforms pure vector search for queries with specific terminology that may not be well-represented in the embedding space. Set alpha to 0.7 as a starting point and adjust based on your document domain and query patterns.
Choosing Between Weaviate and Pinecone
The choice is primarily operational. If your team has infrastructure capacity and wants on-premises deployment, data residency control, or maximum cost efficiency at scale, Weaviate self-hosted (Docker or Kubernetes) is the better fit. Weaviate’s native hybrid search is also more mature and better documented than Pinecone’s sparse-dense approach. If your team wants zero infrastructure management and is willing to pay the managed service premium, Pinecone’s simplicity and reliability are genuine advantages. Both provide production-grade performance for most RAG workloads. The Pinecone API is simpler and the documentation is excellent for getting started quickly. Weaviate’s schema system, multi-tenancy, and self-hosting option provide more flexibility for complex production requirements. For a new RAG project starting fresh with no existing infrastructure, Pinecone gets you to a working production system faster. For an existing platform team with infrastructure experience that wants more control and lower long-term cost, Weaviate self-hosted is the better long-term choice.
Migration Path: ChromaDB to Production
When your ChromaDB prototype is ready to graduate to production, the migration is straightforward because the core logic — embed query, retrieve chunks, pass to LLM — does not change. The only changes are the client library and the vector storage backend. Extract your document chunks and their metadata from ChromaDB using collection.get(), re-embed and upsert into Weaviate or Pinecone in batches, update your retrieval function to use the new client, and run your test suite to confirm retrieval quality is maintained. A complete migration for a collection of a few hundred thousand chunks takes a few hours of compute time for re-embedding plus an afternoon of code changes. The most important step is running retrieval quality benchmarks before and after migration — different backends use slightly different similarity computation defaults, and confirming quality is maintained gives you confidence before switching production traffic.
Weaviate Schema Design for RAG
Weaviate’s schema system requires more upfront thinking than ChromaDB’s schemaless approach, but the structure pays dividends in query flexibility and data integrity. The key design decisions: which properties to store alongside the vector, what data types to use (TEXT for content, INT for numeric metadata like page numbers, DATE for timestamps, BOOLEAN for flags), and whether to use cross-references between collections (Weaviate’s equivalent of a foreign key). For a simple RAG collection, the schema is straightforward — content as TEXT, source as TEXT, chunk index as INT, and any other metadata fields you need for filtering. For complex RAG systems with multiple document types and relationships between them, Weaviate’s cross-reference system lets you model documents containing chapters containing paragraphs with explicit links between them, enabling graph-like traversal within the vector database. This is significantly more powerful than flat metadata in ChromaDB or Pinecone, and it begins to overlap with GraphRAG’s relationship-aware retrieval at the database level rather than requiring a separate graph engine.
Monitoring Production Vector Databases
Production RAG systems need monitoring beyond basic uptime checks. The metrics that matter: query latency at the 95th and 99th percentile (slow retrieval degrades the user experience even if average latency is acceptable), recall quality over time (retrieval quality can degrade if document coverage changes or if query patterns shift away from the indexed content distribution), index size and growth rate (to plan for storage scaling), and error rates on upsert (failed document ingestion leads to missing content in retrieval). Weaviate exposes Prometheus metrics at its /metrics endpoint, suitable for Grafana dashboards. Pinecone provides a usage dashboard in its console with index size, operation counts, and latency metrics. Building a simple nightly evaluation job that runs a fixed set of test queries against your production index and checks whether the top retrieved chunks match expectations is the most reliable way to detect retrieval quality degradation before users notice it. This evaluation dataset should be built from real user queries — the questions your users actually ask are the most valuable benchmark for your specific RAG system.
Cost Comparison: Self-Hosted vs Managed at Scale
Understanding the cost economics helps make the right infrastructure choice. For a RAG system with 10 million document chunks (roughly 50,000 pages of text) and 10,000 queries per day: self-hosted Weaviate on a cloud VM (16GB RAM, 4 vCPU) costs approximately $100-150 per month in compute. Pinecone at this scale (10M vectors, Standard plan) costs approximately $700-900 per month at current pricing. The cost gap widens at scale. At 100M vectors, self-hosted Weaviate on appropriately sized hardware (64GB RAM) costs $400-600 per month versus Pinecone’s significantly higher managed cost. The break-even point where self-hosting becomes more economical than Pinecone is typically around 5-10M vectors, accounting for the engineering time required to operate Weaviate. For organisations with DevOps capacity, self-hosted Weaviate saves meaningful money at production scale. For organisations that prioritise engineering time over infrastructure cost, Pinecone’s managed model may still be worth the premium. Weaviate Cloud (Weaviate’s managed offering) sits between the two: more affordable than Pinecone at scale while retaining the Weaviate feature set without self-hosting overhead.
Getting Started: Which to Choose First
For developers who have not used either, start with Pinecone for the faster onboarding experience. The API is clean, the documentation is excellent, and a working production-grade RAG system is achievable in an afternoon. Once you understand the vector database layer well from the Pinecone perspective, evaluating Weaviate self-hosted becomes easier — the concepts transfer directly and the differences (schema, multi-tenancy, hybrid search configuration) are clear in context. For developers already comfortable with Docker and infrastructure operations who know they need self-hosted deployment, start with Weaviate directly — the Docker setup takes an hour and the Python client is well-designed. Either way, the core RAG pipeline logic remains the same; only the vector storage backend changes between them.