Why Memory Is the Hard Problem in AI Agents LLMs are stateless by default — every request arrives without any memory of previous interactions unless the relevant history is explicitly included in the context. For simple chatbots, this is manageable: include the last N conversation turns and the user’s question feels contextually grounded. For AI … Read more
The Case for Intelligent Routing Most LLM applications use a single model for every request. This is the simplest architecture and the right starting point — but it is almost always suboptimal once you have enough production traffic to analyse. The request distribution of any real application is highly heterogeneous: some requests are simple and … Read more
Why Gemini Flash Matters Google’s Gemini Flash family occupies a specific and valuable position in the LLM market: frontier-adjacent quality at economy pricing, with one of the largest context windows available and native multimodal capabilities included at no additional cost. For developers building cost-sensitive production applications, Gemini Flash is worth serious evaluation alongside the more … Read more
The Startup Advantage in AI Startups have a structural advantage in AI adoption that large enterprises do not: the absence of legacy systems, incumbent processes, and organisational inertia that slow enterprise AI deployment. A ten-person startup can ship an AI-powered product feature in a week that would take a large enterprise six months of approval … Read more
Why AI Governance Has Become Urgent For most of the first decade of enterprise AI, governance was optional. Small AI projects had limited scope, limited impact, and limited visibility — the risk of a poorly governed machine learning model was manageable. That is no longer true. LLMs are now embedded in customer-facing products, internal decision-making … Read more
The System Prompt as Product The system prompt is the most important piece of text in your LLM application. It establishes the model’s persona, defines its scope and constraints, shapes its tone and format, provides essential context, and sets the rules for how it handles the full range of inputs it will encounter. A well-engineered … Read more
The Hallucination Problem in Production Hallucination — LLMs generating confident, fluent text that is factually wrong — is the most discussed quality problem in LLM deployment and, in practice, one of the most manageable once you understand its causes and patterns. The framing of hallucination as a random, unpredictable failure mode is misleading. Hallucination follows … Read more
Why LLM Observability Is Different from Traditional Monitoring Monitoring a traditional API service means tracking uptime, latency, error rates, and throughput. These metrics matter for LLM services too, but they capture only a fraction of what can go wrong. An LLM endpoint can return HTTP 200 with low latency and still produce responses that are … Read more
The Customer Support Opportunity Customer support is one of the highest-volume, most measurable, and most immediately impactful applications of LLMs in enterprise. The use case is clear: a significant fraction of support tickets are repetitive, well-defined queries that follow predictable patterns and can be resolved with information that exists in documentation. Automating these frees human … Read more
What Embeddings Are and Why They Matter An embedding is a numerical representation of a piece of text — a vector of floating-point numbers that encodes the semantic meaning of the text in a form that mathematics can operate on. Two sentences with similar meanings will have embeddings that are close together in the high-dimensional … Read more
