Awesome Explainable AI (XAI) and Interpretable ML Papers and Resources
This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.
This paper provides an introduction to the social science research into explanations. The author provides 4 major findings: (1) explanations are constrastive, (2) explanations are selected, (3) probabilities probably don't matter, (4) explanations are social. These fit into the general theme that…
An important read for anyone using saliency maps. This paper proposes two experiments to determine whether saliency maps are useful: (1) model parameter randomization test compares maps from trained and untrained models, (2) data randomization test compares maps from models trained on the original…
An in-depth description of XAI focused on technqiues for deep learning.
An analysis of how different heatmap-based saliency methods perform based on experimentation with a generated dataset.
Adaptive semantice inpute sampling for explanation.
Accumulated local effects plot.
Autoencoder Based Approach for Local Interpretability.
High-Precision Model-Agnostic Explanations.
Auditing black-box models.
Bayesian local interpretable model-agnostic explanations.
Break down plots for additive attributions.
Class activation mapping.
Confident interpretation of Bayesian decision tree ensembles.
Centered ICE plot.
Combined multiple models metalearner.
Using sampling and queries to extract rules from trained neural networks.
Contribution propogation.
Extracting decision trees from trained neural networks.
Deep label-specific feature learning for image annotation.
Deep Taylor decomposition.
Explanations of evidence in additive classifiers.
Feature importance ranking measure.
Meaninful perturbations model.
Rule extraction using genetic algorithms.
Explain random forest using decision tree.
Exploring classifiers by randomization.
Gaussian process decisions.
Genetic program to evolve decision trees.
Gradient-weighted Class Activation Mapping.
Generalized gradient-based visual explanations.
Making tree ensembles interpretable.
Individual conditional expectation plots.
Integrated gradients.
Interpreting tree ensembles with inTrees.
Iterative orthoganol feature projection.
Information plane visualization.
Kullback-Leibler Projections based LIME.
Extracting decision trees from trained neural networks.
Rationalizing neural predictions with generator and encoder.
Local Interpretable Model-Agnostic Explanations.
Leave-one covariate out.
Local rule-based explanations.
Accurate intelligibile models with pairwise interactions.
Layer-wise relevance propogation.
Model class reliance.
Model explanation system.
Feature importance measure for non-linear algorithms.
Neural interpretation diagram.
Optimized LIME.
Partition aware local model.
Prediction Difference Analysis: Visualize deep neural network decisions.
Partial dependence plots.
Positional oligomer importance matrices for understanding SVM signal detectors.
Transfer information from deep network to simpler model.
Interactive partial dependence diagnostics.
Quantitative input influence.
Extracting symbolic rules from trained neural network ensembles.
Reverse time attention model.
Randomized input sampling for explanation.
Reverse engineering neural networks for rule extraction.
A unified approach to interpretting model predictions.
Similarity, difference, and uniqueness input perturbation.
Visualizing CNN classes.
Programs as black-box explanations.
Interpreting models via Single Tree Approximation.
Explanation of individual classifications using game theory.
Rule extraction from support vector machines.
Testing with concept activation vectors.
Interpretable predictions of tree-ensembles via actionable feature tweaking.
Making sense of a forest of trees.
Consistent feature attribute for tree ensembles.
Feature-space partitioning.
Extracting tree-structured representations of trained networks.
Tree space prototypes.
Visual back-propagation.
Variable effect characteristic curve.
Variable interaction network.
Adapted etraction of comprehensible decision tree in ANNs.
Show, attend, tell attention model.
Like a decision tree with no branches.
The tree provides an interpretation.
Method that predicts based on learned vector graphs of features.
The prototypical clustering method.
Easily plottable and understandable regression.
Easily plottable and understandable classification.
Good classification, poor estimation using conditional probabilities.
Sparse linear model as decision rules including feature interactions.
Authors perform a series of NLP experiments which argue attention does not provide meaningful explanations. They also demosntrate that different attentions can generate similar model outputs.
This is a rebutal to the above paper. Authors argue that multiple explanations can be valid and that the and that attention can produce a valid explanation, if not -the- valid explanation.
Authors argue that addditive explanations (e.g. LIME, SHAP, Break Down) fail to take feature ineractions into account and are thus unreliable.
Authors demonstrate why permuting features is misleading, especially where there is strong feature dependence. They offer several previously described alternatives.
Authors present a number of issues with explainable ML and challenges to interpretable ML: (1) constructing optimal logical models, (2) constructing optimal sparse scoring systems, (3) defining interpretability and creating methods for specific methods. They also offer an argument for why…
Authors demonstrate how saliency methods vary attribution when adding a constant shift to the input data. They argue that methods should fulfill input invariance, that a saliency method mirror the sensistivity of the model with respect to transformations of the input.
A toolkit for XAI which is focused exclusively on tabular data. It implements a variety of data and model evaluation techniques.
SHAP and LIME-based front-end explainer.
A tool for Tensorboard or Notebooks which allows investigating model performance and fairness.
A Python module for using Shapley Additive Explanations.
A debate on whether interpretability is necessary for ML with Rich Caruana and Patrice Simard for and Kilian Weinberger and Yann LeCun against.
A UK-based research center that performs research into ethical AI/ML, which frequently involves XAI.
One of the preeminent researchers in XAI.
The man behind Explainable Boosting Machines.
hesreallyhim/awesome-claude-code
A hand-picked collection of the finest of resources for the most awesome of agents, Claude Code, the undisputed champion of coding companions, from the unstoppable team…
VoltAgent/awesome-agent-skills
A curated collection of 1000+ agent skills from official dev teams and the community, compatible with Claude Code, Codex, Gemini CLI, Cursor, and more.
josephmisiti/awesome-machine-learning
A curated list of awesome Machine Learning frameworks, libraries and software.
EthicalML/awesome-production-machine-learning
A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
academic/awesome-datascience
:memo: An awesome Data Science repository to learn and apply for real world problems.
analysis-tools-dev/static-analysis
⚙️ A curated list of static analysis (SAST) tools and linters for all programming languages, config files, build tools, and more. The focus is on tools which improve…