On Contrastive Explanations for Tree-Based Classifiers

Supplementary material

Contents

• sources: The source code is built on the basis of a GIT clone of https://github.com/crillab/pyxai:
• sources/pyxai/examples/RF/xp-ijcai.py: The software used to compute contrastive explanations.
• [Datasets Link]: The 20 datasets converted and used in our experiments.
More precisely, for each dataset, we have:
• <dataset>.csv: The converted dataset.
• <dataset>.types: A JSON file containing all the information about the features.
• sources/pyxai/examples/Converters/: The source code to convert each dataset:
  • Selection of numerical and categorical features,
  • One-hot encoding of categorical features,
  • Using the OneVsRest method to transform labels into binary classes.
• logs: The outputs produced by the algorithm run in the experiments.
• proofs.pdf: The proofs of the propositions provided in the paper.
• datasets_information.pdf: Additional information about datasets (the number of features, numerical features, categorical features and Boolean features and instances, and then the link to the dataset website). These links allow to get the 20 datasets in their raw form.

Software

Setup

• Be sure to use a Linux OS and to use a version of Python >= to 3.8.
• Create and activate a new virtual environment:
• sudo apt-get install python3.8-venv
• python3 -m venv env
• source env/bin/activate
• Install dependencies:
• python3 -m pip install -r sources/pyxai/requirements.txt

How to use our program

• For a given dataset, the deterministic program sources/pyxai/examples/RF/xp-ijcai.py achieves a 10-fold cross validation process (training/test set separation, random forests computation and accuracies measurement), and then computes contrastive explanations for 100 instances picked up uniformly at random in the test set.
• The instructions are executed from the sources/pyxai/ directory, so the python path must be configured in this way:
• cd sources/pyxai/
• export PYTHONPATH="${PYTHONPATH}:${PWD}/.."
• The next command runs the algorithm on the set of 100 test instances, with a timeout of 100 seconds per instance in order to produce 100 contrastive explanations using the theory defined in the paper (you can download the datasets at [Datasets Link]). In this example, we have extracted the dataset in the sources/pyxai/examples/datasets_converted/ directory.

python3 examples/RF/xp-ijcai.py -dataset=examples/datasets_converted/melb_data_0.csv -types=examples/datasets_converted/melb_data_0.types


• Since our program is deterministic, the results produced for a given dataset must be the same ones as those reported in the logs folder.
• For each instance for which a contrastive explanation is derived, a trace of the computation, similar to the following one, is printed:

           Instance: 1
           Length of the binary representation: 28468
           Number of hard clauses in the CNF encoding the random forest: 70408
           Number of hard clauses in the theory: 58026
           Number of soft clauses due to binary representation: 47
           Number of new soft clauses with new variable due to the theory: 14
           Total number of hard clauses: 128434
           Total number of soft clauses: 61
           Time: 10.66153359413147
           Binary variables to change: 561
           Total features to change: 1
           Is contrastive?: True
      

  This trace gives various information about the size of the encoding used and the computation achieved, including the time needed to generate the contrastive explanation that is returned (on the example, 10.66153359413147s) (the value -1 is used if a timeout has been reached), the number of binary variables in this explanation (on the example, 561) to be changed to get a contrastive explanation based on the set of attributes of the random forest, and the related number of features (from the initial set of attributes) involved in this change (on the example, 1). A checker has also been implemented to verify that the explanation that is generated actually is a contrastive explanation All the outputs produced by our algorithm for computing contrastive explanations are available in the .log files.