How to Interpret XGBoost SHAP Values

by

Feature importance scores built into XGBoost — gain, weight, cover — tell you which features the model used, but not how or in which direction. SHAP values do both: they assign each feature a contribution to each individual prediction, with sign indicating direction and magnitude indicating impact. The result is a local explanation you can read for a single row and a global picture of model behaviour you can summarise across the whole dataset. This guide covers how to compute and interpret both.

Installing SHAP

pip install shap xgboost

Computing SHAP Values for an XGBoost Model

import shap
import xgboost as xgb
from sklearn.model_selection import train_test_split

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

model = xgb.XGBClassifier(n_estimators=500, learning_rate=0.05, max_depth=6,
                           random_state=42, verbosity=0)
model.fit(X_train, y_train)

# TreeExplainer is optimised for tree-based models — much faster than KernelExplainer
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_val)

print(f"SHAP values shape: {shap_values.shape}")  # (n_samples, n_features)
print(f"SHAP for first row: {shap_values[0]}")    # one value per feature

What SHAP Values Mean

Each SHAP value answers: “how much did this feature push this prediction away from the average prediction?” A positive SHAP value means the feature increased the model’s output (pushed toward positive class for a classifier). A negative SHAP value means it decreased the output. The sum of all SHAP values for a row, plus the base value (average model output over the training set), equals the model’s raw output for that row. This additivity is what makes SHAP mathematically rigorous — it is not an approximation but an exact decomposition of the model’s output.

import numpy as np

# Verify additivity: sum of SHAP values + base value = model raw output
base_value = explainer.expected_value   # average log-odds over training set
shap_sum = shap_values[0].sum()
model_output = model.predict(X_val.iloc[[0]], output_margin=True)[0]

print(f"Base value:   {base_value:.4f}")
print(f"SHAP sum:     {shap_sum:.4f}")
print(f"Total:        {base_value + shap_sum:.4f}")
print(f"Model output: {model_output:.4f}")  # should match total

Summary Plot: Global Feature Importance

The summary plot shows every feature’s SHAP values across all validation rows — direction, spread, and feature value correlation at once. Each dot is one row; colour indicates the feature’s actual value (red = high, blue = low); horizontal position is the SHAP value (right = positive impact, left = negative). It replaces a bar chart of importance scores with a complete picture of how each feature behaves across the dataset.

import matplotlib.pyplot as plt

# Summary plot — most informative first look
shap.summary_plot(shap_values, X_val, plot_type='dot', max_display=20)

# Bar version — just mean absolute SHAP (simpler, less informative)
shap.summary_plot(shap_values, X_val, plot_type='bar', max_display=20)

Force Plot: Explaining a Single Prediction

The force plot shows exactly which features pushed a specific prediction up or down from the base value. Red arrows push the prediction higher; blue arrows push it lower. The width of each arrow is proportional to the feature’s SHAP value magnitude. This is the most useful visualisation for explaining an individual decision to a stakeholder.

# Explain first validation row
shap.initjs()  # required for inline JavaScript in Jupyter
force_plot = shap.force_plot(
    explainer.expected_value,
    shap_values[0],
    X_val.iloc[0],
    feature_names=X_val.columns.tolist()
)
shap.save_html('force_plot.html', force_plot)

# Text-based alternative (works outside Jupyter)
print("Top features for row 0:")
row_shap = dict(zip(X_val.columns, shap_values[0]))
for feat, val in sorted(row_shap.items(), key=lambda x: abs(x[1]), reverse=True)[:5]:
    direction = "↑" if val > 0 else "↓"
    print(f"  {feat}: {val:+.4f} {direction}")

Dependence Plot: How One Feature Behaves

A dependence plot shows how the SHAP value for one feature varies with that feature’s actual value, and optionally colours points by a second feature to reveal interactions. It answers the question: “as feature X increases, does its contribution increase, decrease, or do something non-linear?”

# SHAP dependence plot for 'age' feature, coloured by 'income'
shap.dependence_plot('age', shap_values, X_val,
                     interaction_index='income',  # colour by this feature
                     feature_names=X_val.columns.tolist())

# Auto-select the most interacting feature
shap.dependence_plot('age', shap_values, X_val, interaction_index='auto')

Waterfall Plot: Step-by-Step Prediction Explanation

# Requires shap >= 0.40 and Explanation object
explanation = shap.Explanation(
    values=shap_values[0],
    base_values=explainer.expected_value,
    data=X_val.iloc[0].values,
    feature_names=X_val.columns.tolist()
)
shap.waterfall_plot(explanation, max_display=15)

The waterfall plot is a cleaner version of the force plot — it shows the sequential accumulation of SHAP values from base to final prediction, with each bar labelled by feature name and value. Use it when you need a static image for reports or presentations rather than an interactive HTML widget.

SHAP for Multiclass XGBoost

For multiclass models, shap_values returns a list of arrays — one per class. Each array has shape (n_samples, n_features) and contains SHAP values for that class’s raw output score.

model_mc = xgb.XGBClassifier(objective='multi:softmax', num_class=3,
                              n_estimators=300, random_state=42)
model_mc.fit(X_train, y_train)

explainer_mc = shap.TreeExplainer(model_mc)
shap_values_mc = explainer_mc.shap_values(X_val)

print(f"Number of classes: {len(shap_values_mc)}")
print(f"Shape per class: {shap_values_mc[0].shape}")

# Summary plot for class 0
shap.summary_plot(shap_values_mc[0], X_val, title='Class 0 SHAP Values')

Mean Absolute SHAP: Better Global Importance

Mean absolute SHAP is a better global importance measure than XGBoost’s built-in gain or weight scores because it accounts for the actual contribution to predictions, not just how often or how early a feature was used in splits. It is directly comparable across features on the same scale (prediction units).

import pandas as pd
import numpy as np

# Mean absolute SHAP per feature
mean_abs_shap = pd.Series(
    np.abs(shap_values).mean(axis=0),
    index=X_val.columns
).sort_values(ascending=False)

print(mean_abs_shap.head(10))

# Compare with built-in importance
builtin_importance = pd.Series(model.feature_importances_, index=X_val.columns).sort_values(ascending=False)
print("
Built-in importance rank vs SHAP rank:")
for i, feat in enumerate(mean_abs_shap.head(10).index):
    builtin_rank = builtin_importance.index.get_loc(feat) + 1
    print(f"  SHAP rank {i+1}: {feat} (built-in rank {builtin_rank})")

Figure 1 — SHAP value anatomy: each feature pushes the prediction from the base value

Force Plot Anatomy base value age=45 +0.32 income +0.18 debt -0.21 late_pay -0.09 prediction 0.73 approve positive SHAP negative SHAP base = avg prediction base + sum(SHAP) = output

SHAP Interaction Values

SHAP interaction values decompose each SHAP value further into main effects (how a feature contributes on its own) and interaction effects (how pairs of features jointly affect the prediction). They are computationally expensive — O(n_features²) — but useful when you suspect important feature interactions.

# Compute interaction values — slower, more informative
shap_interaction = explainer.shap_interaction_values(X_val[:100])  # use a sample
# Shape: (n_samples, n_features, n_features)
# Diagonal = main effects, off-diagonal = interaction effects

# Plot interactions for top feature
shap.summary_plot(shap_interaction, X_val[:100], max_display=10)

Using SHAP in Production for Explanation Logging

import json

def explain_prediction(model, explainer, row_df):
    """Return top 5 SHAP explanations for a single row as a dict."""
    shap_vals = explainer.shap_values(row_df)[0]
    contributions = dict(zip(row_df.columns, shap_vals))
    top5 = sorted(contributions.items(), key=lambda x: abs(x[1]), reverse=True)[:5]
    return {
        'prediction': float(model.predict_proba(row_df)[0, 1]),
        'base_value': float(explainer.expected_value),
        'top_features': [
            {'feature': k, 'shap': round(float(v), 4),
             'value': round(float(row_df[k].iloc[0]), 4)}
            for k, v in top5
        ]
    }

# Example output logged to database / audit trail
explanation = explain_prediction(model, explainer, X_val.iloc[[0]])
print(json.dumps(explanation, indent=2))

SHAP vs Built-In XGBoost Feature Importance

XGBoost’s built-in importance metrics — gain, weight, cover — measure different things and can produce conflicting rankings. Weight counts how often a feature is used in splits (biased toward high-cardinality features). Gain measures average gain per split (noisy for rarely-used features). Cover measures average sample coverage per split. None of these directly measures contribution to predictions. SHAP measures exactly that — the average impact on model output — making it the most interpretable and consistent importance measure available. When built-in and SHAP rankings disagree significantly, SHAP is almost always more trustworthy. The built-in metrics are useful for debugging (why is a feature not being used at all?) but SHAP is the right tool for communicating feature importance to stakeholders.

Common Misinterpretations

SHAP values are frequently misread in ways that lead to wrong conclusions. The most common: treating SHAP values as causal effects. A positive SHAP for age=45 means the model uses age 45 to increase its output — it does not mean that being 45 years old causes the outcome. The distinction matters when presenting results to stakeholders who may make policy decisions based on feature importance. SHAP describes what the model learned from data, not what drives the outcome in the real world.

A second common mistake: comparing SHAP values across different models. SHAP values are on the scale of the model’s output — log-odds for a logistic-style classifier, raw score for regression. A SHAP of 0.3 from an XGBoost model trained with one set of features is not comparable to a SHAP of 0.3 from a model trained with a different feature set or objective. Always interpret SHAP values relative to the model’s expected_value for that specific model instance, not as absolute scores that travel across models.

Third: using SHAP values computed on training data to interpret test performance. Always compute SHAP values on held-out data. Training-set SHAP values reflect how the model fits the training set, which may be different from how it generalises. On overfit models, training-set SHAP values can look clean and sensible while test-set values reveal the model has learned noise.

Performance: TreeExplainer vs KernelExplainer

For XGBoost, always use shap.TreeExplainer — it runs in polynomial time using the tree structure directly and is typically 100–1000x faster than shap.KernelExplainer, which treats the model as a black box and uses sampling-based approximations. KernelExplainer is model-agnostic and works for any predictor, but its runtime scales poorly with feature count and sample size. For a dataset with 50 features and 10k validation rows, TreeExplainer computes SHAP values in seconds; KernelExplainer takes minutes to hours depending on the number of background samples. The only reason to use KernelExplainer with XGBoost is if TreeExplainer produces unexpected results or if you need SHAP values consistent with a model-agnostic baseline for comparison purposes.

Integrating SHAP into a Model Development Workflow

SHAP is most useful when it becomes a routine part of the development process rather than a one-off explanation step at the end. After training a new model, run the summary plot immediately — it frequently reveals unexpected things: a feature you thought was a strong predictor has nearly zero SHAP magnitude, or a feature you considered noise has strong and consistent positive contributions. These discoveries should feed back into feature engineering, not just be noted and filed away. A feature with high SHAP magnitude but a direction that contradicts domain knowledge is a signal worth investigating — it may be a proxy for a confounding variable, or evidence of leakage.

Before deploying a model, compute SHAP values on a representative sample of production-like inputs and verify the top features make sense to domain experts. This is faster and more informative than manual review of individual predictions, and it catches cases where the model has learned spurious correlations that happen to work on the training distribution but will break on real data. SHAP, used regularly, turns model development from a cycle of training and evaluating metrics into a cycle of training, understanding, and improving — which consistently produces better models and fewer production surprises. Teams that adopt this habit — summary plot after every training run, SHAP sign-off before every deployment — consistently catch more problems earlier and build more trustworthy models.

SHAP transforms XGBoost from a black box into an auditable model. The summary plot gives you a global view of how the model works across the dataset. The force or waterfall plot explains any individual prediction in terms a non-technical stakeholder can follow. Mean absolute SHAP gives you a defensible, model-consistent feature importance ranking. And the interaction values reveal which feature pairs the model has learned to use together. Together they cover everything you need to understand, debug, and explain an XGBoost model in production.

Leave a Comment