mljourney

Gradient accumulation and gradient checkpointing are frequently confused but solve different problems. A precise guide to how each works, when to use them, how to combine them, and how to reason about GPU memory during training.

A practical guide to how attention actually works — scaled dot-product, multi-head, MQA, GQA, Flash Attention, and RoPE — with the implications for memory, throughput, and context length that matter for production deployments.

Handwritten test cases give false confidence. Building an eval dataset from production logs — with stratified sampling, proper labeling, and slice-based reporting — produces evaluations that actually catch regressions before they reach users.

MLflow, W&B, and Neptune all track experiments but optimize for different teams and workflows. A practical comparison across UI quality, self-hosting, hyperparameter optimization, and pricing — with a clear decision framework.

CUDA out of memory errors are almost always solvable without buying more hardware. A practical checklist of GPU memory reduction techniques — gradient checkpointing, 8-bit Adam, LoRA, Flash Attention, and more — in the order to try them.

LoRA, QLoRA, and full fine-tuning each solve the adaptation problem differently. A practical breakdown of memory costs, quality trade-offs, and when each approach is actually the right choice for your hardware and task.

DDP, FSDP, and DeepSpeed ZeRO all distribute training across GPUs but solve different problems. A practical breakdown of when to use each, with memory calculations and a clear decision framework for 7B to 70B model training.

torch.compile delivers 20–50% inference speedup with a single line of code. A practical guide to compilation modes, handling dynamic shapes, combining with Flash Attention and quantization, and knowing when not to use it.