When Do You Need Multiple GPUs? A single GPU handles most LLM workloads up to 34B parameters at Q4. You need multiple GPUs when: the model weights do not fit in a single GPU’s VRAM, you need higher throughput than one GPU delivers for concurrent users, or your training job would take impractically long on … Read more
What the RTX 5090 Brings to LLM Inference The NVIDIA RTX 5090, released in early 2025, is the current flagship consumer GPU based on the Blackwell architecture. For LLM inference specifically, two specifications matter most: it ships with 32 GB of GDDR7 memory — double the 5090’s predecessor, the RTX 4090 — and delivers memory … Read more
Why GPU Cloud Costs Vary So Much GPU cloud infrastructure is sold through dramatically different business models, and understanding them is key to finding the right cost-performance fit. The major hyperscalers (AWS, GCP, Azure) charge premium prices for the ecosystem value they provide — managed services, compliance certifications, SLAs, and deep integration with the rest … Read more
The Three Components of LLM Latency LLM inference latency has three distinct components that require different optimisation strategies. Time to first token (TTFT) is the delay between sending a request and receiving the first token of the response — dominated by prefill time, the cost of processing all input tokens. Time per output token (TPOT) … Read more
Why the H100 vs A100 Decision Matters The NVIDIA A100 has been the workhorse GPU for LLM training and inference since 2020. The H100, released in 2022 and widely available by 2023, represents the next generation — faster, more memory-efficient, and with new hardware features specifically designed for transformer workloads. For teams provisioning GPU infrastructure … Read more
Why Apple Silicon Changed Local LLM Inference Before Apple Silicon, running large language models locally meant either accepting painfully slow CPU inference or buying expensive NVIDIA GPU hardware. Apple’s M-series chips changed this with unified memory architecture — CPU, GPU, and Neural Engine share a single high-bandwidth memory pool. A Mac Studio with 128 GB … Read more
Why Fine-Tune an LLM? A base or instruction-tuned LLM is a generalist — it can do many things reasonably well but few things exceptionally well for your specific domain. Fine-tuning trains the model on examples from your use case so it learns your terminology, output format, tone, and task patterns. The result is a model … Read more
Why Build a Home AI Server? Running LLMs locally on your laptop is fine for experimentation, but it has real limitations: laptop GPUs are underpowered for larger models, thermal throttling hurts sustained performance, and you cannot easily share access with others on your network. A dedicated home AI server solves all of this. It runs … Read more
Why LLM API Costs Matter More Than You Think When teams first start using LLM APIs, cost is rarely the primary concern — getting things working takes priority. But as applications mature and traffic grows, API costs can become a significant operational expense surprisingly quickly. A chatbot handling 10,000 conversations per day at an average … Read more
What Is Vertex AI and Why Use It for LLMs? Vertex AI is Google Cloud’s unified machine learning platform. For LLM workloads, it offers two distinct things: access to Google’s own Gemini models through the Vertex AI API, and infrastructure for deploying any LLM (including open-source models like Llama and Mistral) on managed GPU instances. … Read more
