- permutation importance EXAMPLE!
As the name permutation suggests, this method shuffles the values in a certain column i.e feature and a prediction is made while holding other features as they are. By shuffling the data in such a manner we expect the prediction accuracy to drop as the values are different all together, this procedure is repeated and an average is calculated from the same. The method is applied to all other features and their importance is rated. We have an output which shows the confidence i.e + or - some value of how the feature affects the prediction accuracy.
Output from the coding example above:
As from the screenshot above we can see the most usefull feature to the least useful in that sequence
-
pandas
-
eli5
-
sklearn
Partial dependence plots demonstrate how a property influences the prediction. Partial dependency charts, like permutation importance, are computed after a model has been fitted. The model is fitted to real-world data that has not been modified in any manner.
Output example of a pdp plot, more information about it in the link above.
-
pandas
-
sklearn
-
matplotlib
-
pdpbox
SHAP Values (an acronym from SHapley Additive exPlanations) break down a prediction to show the impact of each feature.
- shap
- pandas
- numpy