The Benchmark Problem Benchmark rankings are the most visible metric in LLM evaluation, and the least useful for choosing a model for a specific production application. MMLU, HumanEval, GSM8K, and their successors measure performance on curated academic test sets under standardised conditions. They tell you how a model performs on a specific type of question, … Read more
The Long Context Revolution Context windows have grown from 4,096 tokens in GPT-3 to 1 million tokens in Gemini 1.5 and 200,000 tokens in Claude. This is not an incremental improvement — it represents a qualitative shift in what LLM applications can do. Tasks that previously required complex chunking pipelines, multi-step summarisation, or external retrieval … Read more
The Unique Security Challenge of LLM Applications LLM applications introduce attack vectors that traditional application security does not address. The core problem is that LLMs process natural language from untrusted sources and generate actions or outputs based on that processing — creating a channel through which adversarial inputs can influence system behaviour in ways that … Read more
The Fine-Tuning Temptation Fine-tuning is the most over-prescribed solution in LLM application development. When a model produces outputs that do not match expectations, the intuitive response is to train it on better examples. But in the majority of cases, the problem is not that the model lacks the capability — it is that the instructions … Read more
Why Model Selection Is Harder Than It Looks The LLM market in 2026 offers dozens of capable models across every price point, capability level, and deployment mode. Benchmark leaderboards exist, but they measure performance on standardised academic tests that may have little correlation with performance on your specific task. A model that ranks first on … Read more
The Decision That Shapes Everything Downstream Choosing between open-source and proprietary LLMs is not primarily a technical decision — it is a strategic one that determines your cost structure, your data governance posture, your dependency on external vendors, your ability to customise, and the ceiling on what you can build. Get it wrong and you … Read more
Beyond Code Completion: The Full Picture Most discussions of LLMs in software engineering focus on code completion tools like GitHub Copilot. These are genuinely useful, but they represent only a fraction of where LLMs are transforming how engineering teams work. The broader impact spans the entire software development lifecycle: requirements analysis, architecture decision-making, code review, … Read more
Why Sales Is a Natural Fit for LLMs Sales work is built on three capabilities that LLMs directly augment: knowing your prospect well, communicating persuasively, and managing a complex pipeline efficiently. Researching a company before a call, drafting personalised outreach, preparing tailored proposals, and writing follow-up communications are all time-intensive text tasks. The average sales … Read more
How LLMs Are Reshaping Financial Analysis Financial analysis is one of the most language-intensive professional functions outside of law. Earnings calls, annual reports, regulatory filings, research notes, credit memos, and board presentations are all primarily text — and large language models are transforming how analysts produce, consume, and synthesise each of them. The productivity gains … Read more
