The Decision That Shapes Everything Downstream
Choosing between open-source and proprietary LLMs is not primarily a technical decision — it is a strategic one that determines your cost structure, your data governance posture, your dependency on external vendors, your ability to customise, and the ceiling on what you can build. Get it wrong and you either pay dramatically more than necessary for your use case, or you discover that the privacy or customisation requirements of your application cannot be met by the vendor you chose. Get it right and you have a foundation that scales cleanly as your AI usage grows.
The choice is also not binary. Most sophisticated AI organisations use a portfolio of models: proprietary frontier models for tasks where state-of-the-art capability is required, open-source models for tasks where the capability gap is acceptable and the cost or privacy advantages are significant, and fine-tuned open-source models for domain-specific tasks where a custom model outperforms general-purpose ones. Understanding when each category is the right choice — and building the routing and evaluation infrastructure to make that choice systematically — is the core skill this guide develops.
What Open Source Actually Means for LLMs
The term “open source” is used inconsistently in the LLM space and the distinction matters. Truly open models (Llama 3, Mistral, Qwen 2.5, Falcon) release model weights under licences that permit commercial use, fine-tuning, and deployment. You download the weights and run them wherever you want — your own servers, a cloud VM, an edge device. The model provider has no visibility into your prompts or outputs. Research models (many academic releases) may release weights but under non-commercial licences. “Open” models that only release APIs without weights (GPT-4, Claude, Gemini) are not open source in any meaningful sense — they are commercial APIs where the underlying model is proprietary. When evaluating “open-source” options, always verify the licence terms: does it permit commercial use? Are there usage restrictions based on company size or application type? Llama 3’s licence, for example, restricts use by organisations with over 700 million monthly active users.
The Case for Proprietary Models
Frontier capability. The best proprietary models (GPT-4o, Claude Opus, Gemini Ultra) still lead open-source models on the hardest tasks — complex multi-step reasoning, nuanced instruction following, advanced coding, and subtle judgment calls. The gap has narrowed considerably in 2026 as open-source models have improved rapidly, but it has not closed for the most demanding use cases. If your application requires the best available reasoning quality and quality materially affects outcomes, proprietary frontier models remain the right choice.
Zero operational overhead. Using a proprietary API means no GPU infrastructure, no model serving infrastructure, no operational team, no on-call rotation for model service issues. The vendor handles all of this. For organisations without ML infrastructure expertise, this operational simplicity is genuinely valuable — the total cost of proprietary APIs often compares favourably to self-hosted open-source when infrastructure and operational costs are properly accounted for.
Fastest access to improvements. When OpenAI releases GPT-4o, Anthropic releases Claude Sonnet 4, or Google releases Gemini Flash 2, you get the improvement by changing a model name in your code. No redeployment, no retraining, no infrastructure changes. For teams that prioritise staying at the frontier of capability, this automatic improvement is valuable.
Managed safety and alignment. Proprietary model providers invest heavily in safety training, content filtering, and alignment. For applications where the cost of generating harmful content is high — customer-facing products, educational tools, healthcare applications — this managed safety layer provides meaningful protection without requiring custom safety infrastructure.
The Case for Open-Source Models
Data privacy and sovereignty. The strongest argument for open-source models is that no data leaves your infrastructure. For regulated industries (healthcare, finance, legal, government), for applications processing genuinely sensitive information, or for organisations with strict data residency requirements, self-hosted open-source models are the only viable option. No vendor agreement, no DPA, no data retention policy, no risk of your prompts being seen by a third party — the data simply never leaves your environment.
Cost at scale. Proprietary API costs are meaningful at production scale. Claude Sonnet at $3/$15 per million tokens, GPT-4o at $5/$15 per million tokens — a high-volume application making millions of calls per day generates significant ongoing costs. A self-hosted Llama or Mistral model on a rented GPU cluster can reduce per-token costs by 80–95% at sufficient scale. The break-even point between API costs and self-hosted infrastructure costs typically falls somewhere between 50 million and 200 million tokens per month depending on the model and hardware configuration.
Full customisation through fine-tuning. With open-source weights, you can fine-tune on your proprietary data to create a model that outperforms general-purpose models on your specific domain. A fine-tuned Llama model trained on your company’s documentation, coding standards, or domain-specific examples will typically outperform a general-purpose frontier model on that specific task, at a fraction of the inference cost. This customisation is simply not available with closed-weight proprietary models where you can only influence the model through prompting.
No vendor dependency. Proprietary API providers can change pricing, deprecate models, alter terms of service, or face outages. Building critical applications on a single proprietary provider creates concentration risk. Open-source models can be downloaded, pinned to a specific version, and deployed without any ongoing vendor relationship. The model you deploy today will behave identically in three years unless you choose to update it.
The Leading Open-Source Models in 2026
Llama 3.3 70B is Meta’s flagship open-source model and the strongest general-purpose open-source option for most tasks. At Q4 quantisation it runs on two 24GB GPUs or a single 80GB A100. It matches or exceeds GPT-4o mini on most benchmarks and is the community-standard model for comparison.
Qwen 2.5 72B from Alibaba is particularly strong on coding and multilingual tasks, outperforming Llama on several benchmarks. It has strong Chinese language capability and is widely used in Asia-Pacific deployments.
Mistral Large 2 offers strong reasoning at moderate size and is the preferred choice for European deployments given Mistral AI’s EU headquarters and commitment to European data governance.
Phi-4 14B from Microsoft is the standout small model — 14 billion parameters with reasoning quality that matches or exceeds 70B models on structured tasks, fitting on a single consumer GPU. Ideal for edge deployment and cost-sensitive inference.
DeepSeek V3 / R1 from a Chinese AI lab delivered a significant capability surprise in early 2025, with performance comparable to GPT-4o on many benchmarks at dramatically lower training cost. R1’s reasoning capability is particularly strong. Widely adopted but raises data governance questions for organisations with strict data sovereignty requirements given its origin.
A Decision Framework
Four questions determine the right choice for each use case. First: does the data contain PII, regulated information, or anything that cannot leave your infrastructure? If yes, the answer is self-hosted open-source — no proprietary API can satisfy this requirement. Second: does the task require frontier reasoning capability where the gap between open and proprietary is material? If yes for a small fraction of your traffic, use a hybrid: open-source for the majority, proprietary frontier for the hard cases. Third: what is your monthly token volume? Below 50 million tokens, proprietary APIs are almost certainly cheaper than self-hosted infrastructure. Above 200 million tokens, self-hosted open-source almost certainly wins on cost. Between 50–200 million tokens, model the full-cost comparison including GPU costs, engineering overhead, and operational maintenance. Fourth: do you have, or can you build, the infrastructure expertise to deploy and operate a self-hosted LLM reliably? If not, proprietary APIs are your practical option regardless of the other factors — a self-hosted model that is unreliable or poorly optimised delivers worse value than a well-managed proprietary API.
Figure 1 — Open Source vs. Proprietary: Decision Matrix
The Hybrid Approach: Best of Both
The most cost-effective and capability-maximising approach for most organisations is a hybrid architecture: route tasks by their requirements rather than using a single model for everything. Simple, high-volume tasks (classification, extraction, summarisation of non-sensitive content) go to a cheap open-source model running on your own infrastructure. Complex reasoning tasks where quality is critical go to a proprietary frontier model API. Sensitive data tasks go to a self-hosted model regardless of complexity. This routing can be implemented as a simple classification layer — the model selection decision itself is inexpensive — and the total cost profile is dramatically better than using either approach exclusively. Building the evaluation infrastructure to measure quality across different models for your specific task types is the investment that makes this routing defensible rather than arbitrary.
Total Cost of Ownership: The Full Comparison
Proprietary API cost comparisons that count only API fees vs. GPU rental costs consistently underestimate the true cost of self-hosted open-source. The full cost of self-hosted deployment includes: GPU infrastructure (purchase or rental), model serving software setup and maintenance, inference optimisation (quantisation, batching, caching), monitoring and observability, security and access control, on-call responsibilities for service reliability, and engineering time for all of the above. A realistic estimate for a small team running a self-hosted Llama 70B deployment in production adds $80,000–$150,000 per year in engineering and operational costs on top of the raw compute. That context changes the break-even calculation significantly — the monthly token volume required before self-hosted is cheaper than proprietary APIs is often considerably higher than naive compute-only comparisons suggest. Do the full-cost calculation for your specific situation before concluding that self-hosted is the cheaper option.