Removes ACV from shapash and fixes dependencies

README.md

@@ -42,7 +42,6 @@
 | 2.0.x | Refactoring Shapash <br> | Refactoring attributes of compile methods and init. Refactoring implementation for new backends | [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/modular.png" width="50" title="modular">](https://github.com/MAIF/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb)
 | 1.7.x | Variabilize Colors <br> | Giving possibility to have your own colour palette for outputs adapted to your design | [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/variabilize-colors.png" width="50" title="variabilize-colors">](https://github.com/MAIF/shapash/blob/master/tutorial/common/tuto-common02-colors.ipynb)
 | 1.6.x | Explainability Quality Metrics <br> [Article](https://towardsdatascience.com/building-confidence-on-explainability-methods-66b9ee575514) | To help increase confidence in explainability methods, you can evaluate the relevance of your explainability using 3 metrics: **Stability**, **Consistency** and **Compacity** | [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/quality-metrics.png" width="50" title="quality-metrics">](https://github.com/MAIF/shapash/blob/master/tutorial/explainability_quality/tuto-quality01-Builing-confidence-explainability.ipynb) 
-| 1.5.x | ACV Backend <br> | A new way of estimating Shapley values using ACV. [More info about ACV here](https://towardsdatascience.com/the-right-way-to-compute-your-shapley-values-cfea30509254). | [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/wheel.png" width="50" title="wheel-acv-backend">](tutorial/explainer_and_backend/tuto-expl03-Shapash-acv-backend.ipynb) |
 | 1.4.x | Groups of features <br> [Demo](https://shapash-demo2.ossbymaif.fr/) | You can now regroup features that share common properties together. <br>This option can be useful if your model has a lot of features. | [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/groups_features.gif" width="120" title="groups-features">](https://github.com/MAIF/shapash/blob/master/tutorial/common/tuto-common01-groups_of_features.ipynb) | 
 | 1.3.x | Shapash Report <br> [Demo](https://shapash.readthedocs.io/en/latest/report.html) | A standalone HTML report that constitutes a basis of an audit document. | [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/report-icon.png" width="50" title="shapash-report">](https://github.com/MAIF/shapash/blob/master/tutorial/generate_report/tuto-shapash-report01.ipynb) | 
 
@@ -287,7 +286,6 @@ This github repository offers many tutorials to allow you to easily get started
 
 - [Compute Shapley Contributions using **Shap**](tutorial/explainer_and_backend/tuto-expl01-Shapash-Viz-using-Shap-contributions.ipynb)
 - [Use **Lime** to compute local explanation, Summarize-it with **Shapash**](tutorial/explainer_and_backend/tuto-expl02-Shapash-Viz-using-Lime-contributions.ipynb)
-- [Use **ACV backend** to compute Active Shapley Values and SDP global importance](tutorial/explainer_and_backend/tuto-expl03-Shapash-acv-backend.ipynb)
 - [Compile faster Lime and consistency of contributions](tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb)
 - [Use **FastTreeSHAP** or add contributions from another backend](tutorial/explainer_and_backend/tuto-expl05-Shapash-using-Fasttreeshap.ipynb)
 - [Use Class Shapash Backend](tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb)

docs/index.html

@@ -64,7 +64,7 @@
  <div class="details">
  <h1>Features</h1>
  <ul>
- <li>Compatible with Shap, Lime and ACV</li>
+ <li>Compatible with Shap and Lime</li>
  <li>Uses shap backend to display results in a few lines of code</li>
  <li>Encoders objects and features dictionaries used for clear results</li>
  <li>Compatible with category_encoders & Sklearn ColumnTransformer</li>

requirements.dev.txt

@@ -1,5 +1,5 @@
 pip>=23.2.0
-numpy==1.21.6
+numpy>1.18.0
 dash==2.3.1
 catboost>=1.0.1
 category-encoders>=2.6.0
@@ -32,13 +32,12 @@ numba>=0.53.1
 nbconvert>=6.0.7
 papermill>=2.0.0
 matplotlib>=3.3.0
-seaborn>=0.12.2
+seaborn==0.12.2
 scipy>=0.19.1
 notebook>=6.0.0
 jupyter-client<8.0.0
 Jinja2>=2.11.0
 phik>=0.12.0
 skranger>=0.8.0
-acv-exp>=1.2.3
 lime>=0.2.0.0
 regex

setup.py

@@ -44,7 +44,7 @@
  'nbconvert>=6.0.7',
  'papermill>=2.0.0',
  'jupyter-client>=7.4.0',
- 'seaborn>=0.12.2',
+ 'seaborn==0.12.2',
  'notebook',
  'Jinja2>=2.11.0',
  'phik'
@@ -53,7 +53,6 @@
 extras['xgboost'] = ['xgboost>=1.0.0']
 extras['lightgbm'] = ['lightgbm>=2.3.0']
 extras['catboost'] = ['catboost>=1.0.1']
-extras['acv'] = ['acv-exp>=1.2.0']
 extras['lime'] = ['lime>=0.2.0.0']
 
 setup_requirements = ['pytest-runner', ]

shapash/backend/__init__.py

@@ -3,7 +3,6 @@
 
 from .base_backend import BaseBackend
 from .shap_backend import ShapBackend
-from .acv_backend import AcvBackend
 from .lime_backend import LimeBackend
 
 

shapash/decomposition/contributions.py

@@ -29,10 +29,6 @@ def inverse_transform_contributions(contributions, preprocessing=None, agg_colum
  The processing apply to the original data.
  agg_columns : str (default: 'sum')
  Type of aggregation performed. For Shap we want so sum contributions of one hot encoded variables.
- For ACV we want to take any value as ACV computes contributions of coalition of variables (like
- one hot encoded variables) differently from Shap and then give the same value to each variable of the
- coalition. As a result we just need to take the value of one of these variables to get the contribution
- value of the group.
 
  Returns
  -------

shapash/explainer/consistency.py

@@ -33,18 +33,19 @@ def tuning_colorscale(self, values):
  color_scale = list(map(list, (zip(desc_pct_df.values.flatten(), self._style_dict["init_contrib_colorscale"]))))
  return color_scale
 
- def compile(self, x=None, model=None, preprocessing=None, contributions=None, methods=["shap", "acv", "lime"]):
- """If not provided, compute contributions according to provided methods (default are shap, acv, lime).
- If provided, check whether they respect the correct format:
+ def compile(self, contributions, x=None, preprocessing=None):
+ """Check whether the contributions respect the correct format:
  contributions = {"method_name_1": contrib_1, "method_name_2": contrib_2, ...}
  where each contrib_i is a pandas DataFrame
 
  Parameters
  ----------
+ contributions : dict
+ Contributions provided by the user if no compute is required.
+ Format must be {"method_name_1": contrib_1, "method_name_2": contrib_2, ...}
+ where each contrib_i is a pandas DataFrame. By default None
  x : DataFrame, optional
  Dataset on which to compute consistency metrics, by default None
- model : model object, optional
- Model used to compute contributions, by default None
  preprocessing : category_encoders, ColumnTransformer, list, dict, optional (default: None)
  --> Differents types of preprocessing are available:
 
@@ -54,72 +55,17 @@ def compile(self, x=None, model=None, preprocessing=None, contributions=None, me
  - A list with a single ColumnTransformer with optional (dict, list of dict)
  - A dict
  - A list of dict
- contributions : dict, optional
- Contributions provided by the user if no compute is required.
- Format must be {"method_name_1": contrib_1, "method_name_2": contrib_2, ...}
- where each contrib_i is a pandas DataFrame. By default None
- methods : list
- Methods used to compute contributions, by default ["shap", "acv", "lime"]
  """
  self.x = x
  self.preprocessing = preprocessing
- if contributions is None:
- if (self.x is None) or (model is None):
- raise ValueError('If no contributions are provided, parameters "x" and "model" must be defined')
- contributions = self.compute_contributions(self.x, model, methods, self.preprocessing)
- else:
- if not isinstance(contributions, dict):
- raise ValueError('Contributions must be a dictionary')
+ if not isinstance(contributions, dict):
+ raise ValueError('Contributions must be a dictionary')
  self.methods = list(contributions.keys())
  self.weights = list(contributions.values())
 
  self.check_consistency_contributions(self.weights)
  self.index = self.weights[0].index
 
- def compute_contributions(self, x, model, methods, preprocessing):
- """
- Compute contributions based on specified methods
-
- Parameters
- ----------
- x : pandas.DataFrame
- Prediction set.
- IMPORTANT: this should be the raw prediction set, whose values are seen by the end user.
- x is a preprocessed dataset: Shapash can apply the model to it
- model : model object
- Model used to consistency check. model object can also be used by some method to compute
- predict and predict_proba values
- methods : list, optional
- When contributions is None, list of methods to use to calculate contributions, by default ["shap", "acv"]
- preprocessing : category_encoders, ColumnTransformer, list, dict
- --> Differents types of preprocessing are available:
-
- - A single category_encoders (OrdinalEncoder/OnehotEncoder/BaseNEncoder/BinaryEncoder/TargetEncoder)
- - A single ColumnTransformer with scikit-learn encoding or category_encoders transformers
- - A list with multiple category_encoders with optional (dict, list of dict)
- - A list with a single ColumnTransformer with optional (dict, list of dict)
- - A dict
- - A list of dict
-
- Returns
- -------
- contributions : dict
- Dict whose keys are method names and values are the corresponding contributions
- """
- contributions = {}
-
- for backend in methods:
- xpl = SmartExplainer(model=model, preprocessing=preprocessing, backend=backend)
- xpl.compile(x=x)
- if xpl._case == "classification" and len(xpl._classes) == 2:
- contributions[backend] = xpl.contributions[1]
- elif xpl._case == "classification" and len(xpl._classes) > 2:
- raise AssertionError("Multi-class classification is not supported")
- else:
- contributions[backend] = xpl.contributions
-
- return contributions
-
  def check_consistency_contributions(self, weights):
  """
  Assert contributions calculated from different methods are dataframes

shapash/explainer/smart_explainer.py

@@ -42,7 +42,7 @@ class SmartExplainer:
  predict and predict_proba values
  backend : str or shpash.backend object (default: 'shap')
  Select which computation method to use in order to compute contributions
- and feature importance. Possible values are 'shap', 'acv' or 'lime'. Default is 'shap'.
+ and feature importance. Possible values are 'shap' or 'lime'. Default is 'shap'.
  It is also possible to pass a backend class inherited from shpash.backend.BaseBackend.
  preprocessing : category_encoders, ColumnTransformer, list, dict, optional (default: None)
  --> Differents types of preprocessing are available:

shapash/explainer/smart_state.py

@@ -62,10 +62,6 @@ def inverse_transform_contributions(self, contributions, preprocessing, agg_colu
  Single step of preprocessing, typically a category encoder.
  agg_columns : str (default: 'sum')
  Type of aggregation performed. For Shap we want so sum contributions of one hot encoded variables.
- For ACV we want to take any value as ACV computes contributions of coalition of variables (like
- one hot encoded variables) differently from Shap and then give the same value to each variable of the
- coalition. As a result we just need to take the value of one of these variables to get the contribution
- value of the group.
 
  Returns
  -------

shapash/utils/category_encoder_backend.py

@@ -198,10 +198,6 @@ def calc_inv_contrib_ce(x_contrib, encoding, agg_columns):
  The processing apply to the original data.
  agg_columns : str (default: 'sum')
  Type of aggregation performed. For Shap we want so sum contributions of one hot encoded variables.
- For ACV we want to take any value as ACV computes contributions of coalition of variables (like
- one hot encoded variables) differently from Shap and then give the same value to each variable of the
- coalition. As a result we just need to take the value of one of these variables to get the contribution
- value of the group.
 
  Returns
  -------

shapash/utils/columntransformer_backend.py

@@ -195,10 +195,6 @@ def calc_inv_contrib_ct(x_contrib, encoding, agg_columns):
  The processing apply to the original data.
  agg_columns : str (default: 'sum')
  Type of aggregation performed. For Shap we want so sum contributions of one hot encoded variables.
- For ACV we want to take any value as ACV computes contributions of coalition of variables (like
- one hot encoded variables) differently from Shap and then give the same value to each variable of the
- coalition. As a result we just need to take the value of one of these variables to get the contribution
- value of the group.
 
  Returns
  -------

shapash/utils/utils.py

@@ -232,7 +232,7 @@ def compute_sorted_variables_interactions_list_indices(interaction_values):
  for i in range(tmp.shape[0]):
  tmp[i, i:] = 0
 
- interaction_contrib_sorted_indices = np.dstack(np.unravel_index(np.argsort(tmp.ravel()), tmp.shape))[0][::-1]
+ interaction_contrib_sorted_indices = np.dstack(np.unravel_index(np.argsort(tmp.ravel(), kind="stable"), tmp.shape))[0][::-1]
  return interaction_contrib_sorted_indices