Three Approaches, One Question When an LLM does not behave the way you need it to, three distinct strategies are available to improve its performance: prompting (changing what you say to the model at inference time), retrieval-augmented generation (providing relevant external information at inference time), and fine-tuning (changing the model’s weights through training). These are … Read more
The Data Analysis Transformation Data analysis has historically required either technical skill (SQL, Python, R) or expensive analyst time to translate business questions into code and results. LLMs are disrupting both requirements. They can write SQL and Python from natural language descriptions, explain complex data in plain language, generate visualisation code from data summaries, identify … Read more
Why Few-Shot Prompting Still Matters in 2026 With powerful frontier models that follow complex instructions reliably, it is tempting to think that few-shot prompting — providing examples in the prompt — is a technique from an earlier, less capable era. It is not. Few-shot examples remain one of the most powerful tools for eliciting specific … Read more
What AI Pair Programming Actually Is AI pair programming is not autocomplete. Modern LLM coding assistants — GitHub Copilot, Cursor, Claude, ChatGPT — understand code context at the function, file, and increasingly codebase level. They can explain unfamiliar code, suggest approaches to new problems, generate complete implementations from docstrings, write tests for existing functions, refactor … Read more
What Is a Token? Every LLM input is converted into tokens before the model processes it. A token is not a word, not a character, and not a syllable — it is a unit that falls somewhere between characters and words, determined by the tokeniser’s vocabulary. In practice, common English words are typically one token … Read more
The Structured Output Problem LLMs generate free-form text by default. When your application needs the output in a specific format — JSON for an API response, a list with defined fields, a table with precise columns — getting reliable structured output requires deliberate engineering. The naïve approach of asking the model to “respond in JSON” … Read more
Why Context Compression Matters Even with million-token context windows, there are compelling reasons to compress what you send to the model. Cost scales linearly with input tokens — a 100,000-token prompt costs 10x more than a 10,000-token one. Latency scales with input length, directly affecting user experience in interactive applications. And longer contexts do not … Read more
Why LLM Product Strategy Is Different Building products on top of large language models requires a different strategic framework than traditional software products. The underlying capability is non-deterministic, rapidly improving, and shared by competitors — three properties that make standard product moats (proprietary technology, switching costs, network effects) harder to build and maintain. A product … Read more
The Benchmark Problem Benchmark rankings are the most visible metric in LLM evaluation, and the least useful for choosing a model for a specific production application. MMLU, HumanEval, GSM8K, and their successors measure performance on curated academic test sets under standardised conditions. They tell you how a model performs on a specific type of question, … Read more
The Long Context Revolution Context windows have grown from 4,096 tokens in GPT-3 to 1 million tokens in Gemini 1.5 and 200,000 tokens in Claude. This is not an incremental improvement — it represents a qualitative shift in what LLM applications can do. Tasks that previously required complex chunking pipelines, multi-step summarisation, or external retrieval … Read more
