Explainable AI (XAI) with SHAP and PDP.
Delivering business value and deploying machine learning models comes with a responsibility. As a part of it ML models should be interpretable and explainable. They should not be treated as a black box anymore.
Importance of XAI:
Data science teams sometimes finds it difficult to explain the ML models and their behaviour to the business teams. Every business vertical is trying to adopt machine learning techniques. As part of the ML model delivery we should make sure our models more explainable and interpretable. Following are the few things that ML engineers are accountable while delivering the machine learning models to production.
- Accountability
- Trust
- Compliance
- Performance
This is where XAI comes in handy. We can account for each of the above mentioned with explainable AI. There are two approaches of explaining complex machine learning models:
- Model-Agnostic Approach
- Model-Specific Approach
As a part of this discussion we will be mostly looking into model agnostics approaches that too mainly Shapley Additive exPlanations (SHAP) which is based on Shapley values. After that, I am gonna discuss about partial dependence plots.
Shapley Values:
A prediction can be explained by assuming that each feature value of the instance is a “player” in a game where the prediction is the payout. Shapley values — a method from coalition game theory — tells us how to fairly distribute the “payout” among the features.
Shapley values have found applications in several fields. They can be applied in machine learning to explain feature contributions where features are the players or team members and the model prediction is the payout of the game or teamwork.
To calculate the importance of feature j, we can intuitively think of the process as, for each iteration, drawing feature values in random order for all features except for feature j before we then calculate the difference of prediction with and without feature j. The Shapley value is computed by taking the average of difference from all combinations. Essentially, the Shapley value is the average marginal contribution of a feature considering all possible combinations.
The main advantage of Shapley value is that it provides a fair contribution of features with mathematically proven theory and properties about its mechanism. However, its downside is complexity of the algorithm because the combination of features grows exponentially. Therefore, in a practical scenario, the Shapley value can only be estimated using a subset of combinations.
Partial Dependence Plots:
A partial dependence (PD) plot depicts the functional relationship between a small number of input variables and predictions. They show how the predictions partially depend on values of the input variables of interest. PD plots look at the variable of interest across a specified range. At each value of the variable, the model is evaluated for all observations of the other model inputs, and the output is then averaged. Thus, the relationship they depict is only valid if the variable of interest does not interact strongly with other model inputs.
Since variable interactions are common in actual practice, you can use higher-order (such as two-way) partial dependence plots to check for interactions among specific model variables.
Resources for SHAP and PDP:
There are few implementations of above discussed methods for model explainability.
