Imbalanced classification — fraud detection, medical diagnosis, churn prediction, rare event detection — is where default model settings fail most predictably. A model that predicts the majority class for every sample achieves 99% accuracy on a 1:99 dataset while being completely useless. LightGBM has several built-in mechanisms for imbalanced data, and combining them with the right evaluation metrics and decision threshold tuning produces reliable models on datasets where naive approaches fall apart.
Why Accuracy Fails on Imbalanced Data
On a dataset where 1% of samples are positive, a model that always predicts negative achieves 99% accuracy. This is why accuracy is the wrong metric for imbalanced problems — it does not penalise the model for ignoring the minority class entirely. Use AUC-ROC (measures discriminative ability across all thresholds), PR-AUC/Average Precision (more informative than ROC when the positive class is rare), F1 score (harmonic mean of precision and recall, requires choosing a threshold), or recall at a fixed precision (useful when false negatives are costly).
is_unbalance: Automatic Class Weighting
The simplest approach — let LightGBM handle it. Setting is_unbalance=True automatically weights the minority class by the inverse of its frequency. For a 1:99 ratio, the minority class gets weight 99 and the majority class gets weight 1. This makes the model penalise minority class errors proportionally more during training.
import lightgbm as lgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score, average_precision_score
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2,
stratify=y, random_state=42)
model = lgb.LGBMClassifier(
n_estimators=1000,
learning_rate=0.05,
num_leaves=63,
is_unbalance=True, # automatic class weighting
metric='auc',
random_state=42,
verbose=-1
)
model.fit(X_train, y_train,
eval_set=[(X_val, y_val)],
callbacks=[lgb.early_stopping(100), lgb.log_evaluation(200)])
probas = model.predict_proba(X_val)[:, 1]
print(f"AUC-ROC: {roc_auc_score(y_val, probas):.4f}")
print(f"PR-AUC: {average_precision_score(y_val, probas):.4f}")scale_pos_weight: Manual Class Weighting
For more control, set scale_pos_weight to the ratio of negative to positive samples. This is the standard approach when is_unbalance produces suboptimal results — you can tune the ratio as a hyperparameter:
import numpy as np
# Compute class ratio
n_neg = (y_train == 0).sum()
n_pos = (y_train == 1).sum()
ratio = n_neg / n_pos
print(f"Class ratio (neg/pos): {ratio:.1f}")
model = lgb.LGBMClassifier(
n_estimators=1000,
learning_rate=0.05,
num_leaves=63,
scale_pos_weight=ratio, # or tune this as a hyperparameter
metric='auc',
random_state=42, verbose=-1
)
model.fit(X_train, y_train, eval_set=[(X_val, y_val)],
callbacks=[lgb.early_stopping(100)])Stratified Splitting and Cross-Validation
Always use stratified splits with imbalanced data — random splitting can produce folds with zero or near-zero positive samples, especially at severe imbalance ratios. Both train_test_split and StratifiedKFold handle this with stratify=y:
from sklearn.model_selection import StratifiedKFold, cross_val_score
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
model_cv = lgb.LGBMClassifier(is_unbalance=True, metric='auc', verbose=-1)
# Score with AUC-ROC — more informative than accuracy for imbalanced data
auc_scores = cross_val_score(model_cv, X, y, cv=cv, scoring='roc_auc')
ap_scores = cross_val_score(model_cv, X, y, cv=cv, scoring='average_precision')
print(f"AUC-ROC: {auc_scores.mean():.4f} (+/- {auc_scores.std():.4f})")
print(f"PR-AUC: {ap_scores.mean():.4f} (+/- {ap_scores.std():.4f})")Decision Threshold Tuning
By default, LightGBM (like most classifiers) uses 0.5 as the decision threshold. For imbalanced data, the optimal threshold is almost never 0.5. Lowering it catches more positives at the cost of more false positives; raising it reduces false positives at the cost of missing more positives. The right threshold depends on the cost of each type of error in your application.
from sklearn.metrics import precision_recall_curve, f1_score
import numpy as np
probas = model.predict_proba(X_val)[:, 1]
precisions, recalls, thresholds = precision_recall_curve(y_val, probas)
# Find threshold maximising F1
f1_scores = 2 * precisions * recalls / (precisions + recalls + 1e-8)
best_idx = np.argmax(f1_scores[:-1])
best_threshold = thresholds[best_idx]
print(f"Best F1 threshold: {best_threshold:.4f}")
print(f"Precision: {precisions[best_idx]:.4f} Recall: {recalls[best_idx]:.4f}")
# Apply threshold
preds = (probas >= best_threshold).astype(int)
print(f"F1 at best threshold: {f1_score(y_val, preds):.4f}")Figure 1 — Precision-Recall curve: choose threshold based on your cost trade-off
SMOTE: Synthetic Minority Oversampling
SMOTE generates synthetic minority class samples by interpolating between existing minority samples, rather than simply duplicating them. Use it only on training data, never on validation or test data:
from imblearn.over_sampling import SMOTE
smote = SMOTE(random_state=42, k_neighbors=5)
X_resampled, y_resampled = smote.fit_resample(X_train, y_train)
print(f"Original: {y_train.value_counts().to_dict()}")
print(f"Resampled: {pd.Series(y_resampled).value_counts().to_dict()}")
model = lgb.LGBMClassifier(n_estimators=500, metric='auc', verbose=-1)
model.fit(X_resampled, y_resampled, eval_set=[(X_val, y_val)],
callbacks=[lgb.early_stopping(50)])SMOTE works well when the minority class is compact and the decision boundary is relatively smooth. It can hurt performance when minority samples form multiple distinct clusters (interpolating between clusters creates unrealistic synthetic samples) or when the feature space is high-dimensional (interpolation in high dimensions is unreliable). Always compare SMOTE against is_unbalance=True and class weighting on your specific dataset — the best approach varies.
Combining Approaches: The Recommended Stack
The most reliable strategy for imbalanced LightGBM: use is_unbalance=True as a baseline, evaluate with AUC-ROC and PR-AUC, tune the decision threshold based on your precision-recall trade-off requirement, and use stratified cross-validation throughout. Add SMOTE only if the baseline underperforms on the minority class and you have verified it helps on held-out data. Tune scale_pos_weight with Optuna if the automatic weighting from is_unbalance is suboptimal. The combination of class weighting, the right metric, and threshold tuning resolves most imbalanced classification problems without needing complex sampling pipelines.
Choosing the Right Evaluation Metric
The metric you optimise during training and the metric you report should both reflect what matters for your application. AUC-ROC measures how well the model separates classes across all decision thresholds — it is threshold-independent and a good default for most imbalanced problems. However, AUC-ROC can be misleadingly optimistic on severely imbalanced datasets because it includes many threshold values where the model correctly classifies the abundant negative class. PR-AUC (area under the precision-recall curve) is more informative for severe imbalance — it focuses only on the positive class performance and is harder to inflate through correct negative predictions. For operational decisions, F-beta score lets you weight recall more than precision (beta > 1) or vice versa (beta < 1) depending on whether false negatives or false positives are more costly in your domain.
from sklearn.metrics import fbeta_score, classification_report
# F2 score — weights recall twice as much as precision
# Use when false negatives (missed positives) are costly, e.g. fraud or disease
preds = (probas >= best_threshold).astype(int)
f2 = fbeta_score(y_val, preds, beta=2)
print(f"F2 score: {f2:.4f}")
# Full classification report
print(classification_report(y_val, preds, target_names=['negative', 'positive']))Undersampling the Majority Class
Instead of oversampling the minority class, you can undersample the majority class to create a balanced training set. Random undersampling is the simplest approach and surprisingly effective when you have a lot of majority class data — removing 90% of majority samples still leaves enough for the model to learn the decision boundary.
from imblearn.under_sampling import RandomUnderSampler
rus = RandomUnderSampler(sampling_strategy=0.5, random_state=42)
# sampling_strategy=0.5 means minority:majority = 1:2 after resampling
X_resampled, y_resampled = rus.fit_resample(X_train, y_train)
print(pd.Series(y_resampled).value_counts())Undersampling loses information from the majority class, which can hurt if majority class patterns help the model distinguish it from the minority class. The advantage is that training is much faster on the smaller dataset. A common hybrid — Random Undersampling + SMOTE — undersamples the majority class and oversamples the minority class to a moderate imbalance ratio rather than full balance, combining the speed benefit of undersampling with the information preservation of oversampling.
Optuna Tuning for Imbalanced LightGBM
import optuna
from sklearn.model_selection import StratifiedKFold, cross_val_score
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
def objective(trial):
params = {
'n_estimators': trial.suggest_int('n_estimators', 200, 2000),
'learning_rate': trial.suggest_float('lr', 0.005, 0.3, log=True),
'num_leaves': trial.suggest_int('num_leaves', 20, 150),
'min_child_samples': trial.suggest_int('min_child_samples', 20, 500),
'scale_pos_weight': trial.suggest_float('spw', 1.0, n_neg/n_pos * 2),
'subsample': trial.suggest_float('subsample', 0.5, 1.0),
'subsample_freq': 1,
'colsample_bytree': trial.suggest_float('colsample_bytree', 0.5, 1.0),
'reg_lambda': trial.suggest_float('reg_lambda', 1e-2, 10, log=True),
'random_state': 42, 'verbose': -1
}
model = lgb.LGBMClassifier(**params)
scores = cross_val_score(model, X_train, y_train, cv=cv, scoring='average_precision')
return scores.mean()
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=100)
print(study.best_params)Notice that scale_pos_weight is included as an Optuna parameter with a search range up to twice the natural ratio. The natural ratio (n_neg/n_pos) is the starting point but the optimal value depends on the cost structure of your problem — tuning it lets Optuna find the best trade-off between precision and recall for your specific dataset and metric.
Imbalanced classification with LightGBM is largely a problem of metrics and calibration rather than model architecture. The algorithm itself is capable — the challenge is telling it what you care about (via class weights and loss metric), measuring it correctly (AUC-ROC, PR-AUC rather than accuracy), and translating probabilities into decisions at the right threshold for your application. Get those three things right and LightGBM is one of the strongest off-the-shelf choices for imbalanced tabular classification tasks.