中文

InterpretDL is short for interpretation of deep learning models.

It is a model interpretation toolkit for PaddlePaddle models.

It provides many useful interpretation algorithms, like LIME, Grad-CAM, IntergratedGradients, etc.

It also contains (and will contain) SOTA and new interpretation algorithms.

InterpretDL is under active construction and all contributions are welcome!

Interpretation algorithms give a hint of why the black-box model makes this decision.

The following table gives visualizations of several interpretation algorithms applied to the original image with respect to the label "bull_mastiff."

Original Image	Integrated Gradients	SmoothGrad	LIME	Grad-CAM

• Demo
• Installation
  • Pip installation
  • Developer installation
• Documentation
• Usage Guideline
• Contribution
• Roadmap
  • Algorithms
  • Tutorials
• Copyright and License

It requires the deep learning framework paddlepaddle, recommendation with CUDA support.

pip install interpretdl

# or with baidu mirror
pip install interpretdl -i https://mirror.baidu.com/pypi/simple

git clone https://github.com/PaddlePaddle/InterpretDL.git
# ... fix bugs or add new features
python setup.py install
# welcome to propose pull request and contribute

Online link: interpretdl.readthedocs.io.

Or generate the docs locally:

git clone https://github.com/PaddlePaddle/InterpretDL.git
cd docs
make html
open _build/html/index.html

All interpreters inherit the abstract class Interpreter, of which interpret(**kwargs) is the function to call.

# an example of SmoothGradient Interpreter.

# import ...

def paddle_model(data):
    class_num = 1000
    model = ResNet50()
    logits = model.net(input=data, class_dim=class_num)
    probs = fluid.layers.softmax(logits, axis=-1)
    return probs

img_path = 'assets/deer.png'
sg = SmoothGradInterpreter(paddle_model, "assets/ResNet50_pretrained")
gradients = sg.interpret(img_path, visual=True, save_path='sg_test.jpg')

More tutorials will be released.

We are planning to create a useful toolkit for offering the model interpretation.

We are planning to implement the algorithms below (categorized into sensitivity interpreters and algorithmic interpreters):

• LIME
• FastNormLIME
• NormLIME
• LIMEPrior
• SmoothGrad
• Occlusion
• DeepLIFT
• GradientSHAP
• GradCAM
• IntegratedGradients
• InfluenceFunction
• ForgettingEvent
• SGDNoise
• More ...

We plan to provide at least one example for each interpretation algorithm, and hopefully on different applications, as in CV and NLP.

• IntegratedGraients: Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017
• GradCAM: Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017
• SmoothGrad: SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017
• GradientShap: A Unified Approach to Interpreting Model Predictions, Scott M. Lundberg et al. 2017
• Occlusion: Visualizing and Understanding Convolutional Networks
• Lime: "Why Should I Trust You?": Explaining the Predictions of Any Classifier, Marco Tulio Ribeiro et al. 2016
• NormLime: NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks, Isaac Ahern et al. 2019
• ScoreCAM: Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks, Haofan Wang et al. 2020

InterpretDL is provided under the Apache-2.0 license.