[RFC] Add interpretability API as xgboost.interpret module functions

[RAMitchell]

Summary

We have new work underway on shapley values and other related interpretability concepts. This will add new functionality, however the current feature importance/shapley features are included in the predict API. I propose extending the python api with a module (xgboost.interpret) for interpretability, containing stateless functions exposing upcoming features.

These functions accept either a Booster or an sklearn-style XGB* model, plus DMatrix/array-like inputs, and return well-typed results (arrays and or light-weight result objects).

Motivation

• Minimize disruption to existing Booster / sklearn APIs while adding interpretability features.
• Improve discoverability and documentation (module-level functions are easy to document and test).
• Allow incremental implementation: start as wrappers over existing predict(pred_contribs=..., pred_interactions=...), then evolve internals (esp. top-k) without changing the public API.

Proposed public API

Add a new module:

• xgboost/interpret.py

Functions (accept Booster | XGBModel and DMatrix | array-like | pandas):

• shap_values(model, X, *, output_margin=False, iteration_range=None, approx=False, validate_features=True, feature_names=None, return_bias=False)
• shap_interactions(model, X, *, output_margin=False, iteration_range=None, approx=False, validate_features=True, feature_names=None)
• topk_interactions(model, X, *, k=50, metric="mean_abs", per_row=False, output_margin=False, iteration_range=None, approx=False, validate_features=True, feature_names=None)
• partial_dependence(model, X, *, features, grid_resolution=50, percentiles=(0.05,0.95), grid=None, sample=None, random_state=0, output="prediction", iteration_range=None)

Dispatch/behavior notes

• Internally normalize model to a Booster via model being Booster or having get_booster().
• Normalize X to DMatrix if needed; respect feature names where possible.
• Initial SHAP implementations can wrap existing Booster.predict(..., pred_contribs=True/pred_interactions=True) for compatibility.
• topk_interactions should ideally avoid materializing full (n, p, p) tensors; target a C++ implementation to compute aggregated top-k pairs efficiently.

Return types

Prefer lightweight result objects to keep outputs consistent and extensible:

• ShapValues(values, base_values, feature_names, model_output, ...)
• ShapInteractions(values, feature_names, ...) with helpers for main effects / pair extraction
• TopKInteractions(pairs, scores, pair_names=None, per_row=None, ...)
• PDP(features, grid_values, averages, ...)

Documentation plan (Sphinx)

• Add docs/python/interpretability.rst
  • narrative examples + API reference using .. autofunction:: for each function
  • .. autoclass:: for result types

@ron-wettenstein

[RAMitchell]

The proposal for GroupShap #11493 can be folded into this API - we can simply allow relabeling of the features.

[trivialfis]

interpretability, containing stateless functions exposing upcoming features.

Thank you for the RFC. It would be great if you could provide a scope for the new module. For instance, is visualization included?

ShapValuesResult

Maybe omit the "Result" in its name.

target a streaming / C++ implementation to compute aggregated top-k pairs efficiently.

I'm curious about your plan for this, specifically the return values.

[RAMitchell]

My instinct was to avoid visualisation here, but we do already have graphviz and matplotlib as optional dependencies. Maybe we should just go ahead and make it as easy as possible for the user. In any case that can be for a later time - lets focus on returning the raw data for now.

[RAMitchell]

@trivialfis do you prefer this api as a function that accepts a booster/sklearn model or would you prefer adding methods to booster/sklearn model. I find we have too many methods already and I like the idea of being able to cleanly separate the docs and implementations.

[trivialfis]

I think we should clearly separate computation from visualization (agreeing with your preference), in a similar way that's stated by the mlr3 project. So, I prefer having them to be separated from the booster class. Similar design choice can be found in sklearn visualization module: https://scikit-learn.org/stable/visualizations.html see the RocCurveDisplay.from_estimator.

As for whether these should be functions or classes, I don't have a preference. You have a better understanding of potential states and can make better choices than I.