Skip to content

shap instability: additivity check fails #192

Description

@JochenSiegWork

Problem

For the standard case of explaining Random Forest with Morgan fingerprints using SHAP's TreeExplainer regularly leads to additivity check error thrown by shap on different data sets.

Since the problem comes from shap, there is not much we can do to fix this directly. However, a workaround is to set the depth of the trees to a smaller number, like max_depth=32.

From looking through the shap issues it seems that there are multiple different causes for additivity check fails in the shap shap/shap#3992. In our case, we use binary fingerprints that are relatively sparse (around <5% of the values are 1) which could mean that the sparsity is the root of the problem, as described in shap/shap#4042

Here is a related MolPipeline issue: #137

Possible actions

  • Add the max_depth=32 to the XAI example code in the README and the notebook with a note to allow users to obtain explanations.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions