XGBoost’s GPU training can cut training time by 5–50x depending on dataset size and hardware. On a dataset with a million rows the difference between CPU and GPU training is often the difference between waiting 20 minutes per Optuna trial and waiting 45 seconds. This guide covers how to enable GPU training, what the parameter … Read more
Early stopping is one of the most useful features in XGBoost — and one of the most frequently misconfigured. It prevents overfitting by halting training when the validation metric stops improving, and it eliminates the need to manually search for the right number of trees. This guide covers how to use it correctly, the gotchas … Read more
LightGBM hyperparameter tuning guide covering num_leaves, min_child_samples, subsample, regularisation, and Optuna search.
CatBoost quietly became one of the best gradient boosting libraries available. It handles categorical features natively without preprocessing, trains fast on GPU, and consistently outperforms XGBoost and LightGBM on datasets with lots of categorical data — which describes most real-world tabular datasets. If you have been manually one-hot encoding categoricals before throwing them at XGBoost, … Read more
Bi-encoder retrieval — the kind used in standard vector search — is fast but imprecise. It compresses each document into a fixed-size vector and scores query-document similarity independently, without either knowing anything about the other. Cross-encoders flip this: they see the query and document together and output a single relevance score. That joint view is … Read more
Building a RAG pipeline is straightforward. Knowing whether it actually works — and what specifically is broken when it does not — requires systematic evaluation. RAG systems can fail in at least four distinct ways: irrelevant retrieval, faithfulness failures where the LLM ignores the retrieved context, knowledge gaps where the right context is missing, and … Read more
Standard RAG retrieves once and generates once. Corrective RAG (CRAG) and Self-RAG add a step that standard pipelines skip entirely: evaluating whether the retrieved content is actually good enough before generating. Both architectures recognise that retrieval sometimes fails — the retrieved chunks are irrelevant, outdated, or insufficient — and build in mechanisms to detect and … Read more
RAPTOR, HyDE, and RAG Fusion are three advanced retrieval techniques that each address a different fundamental limitation of standard vector RAG. This guide gives each the depth it deserves: the core idea, why it works, when to use it, and a working implementation you can drop into an existing pipeline. RAPTOR: Retrieval at Multiple Levels … Read more
