On the Computation of Example-Based Abductive Explanations for Random Forests

Supplementary material

Contents

• sources: The source code is built on the basis of a GIT clone of https://github.com/crillab/pyxai:
• sources/pyxai-anchored/ijcai2024-anchored/compute_anchored.py: The software used to compute most anchored explanations.
• [Datasets Link]: The 14 datasets converted and used in our experiments.
More precisely, for each dataset, we can find:
• datasets/<dataset>.csv: The converted dataset.
• datasets/<dataset>.types: A JSON file containing all the information about the features.
• models/<dataset>/: A directory per dataset containing the 10 models used (the model number is indicated by <id>):
  • models/<dataset>/<dataset>.<id>.model: The random Forest generated using Sklearn.
  • models/<dataset>/<dataset>.<id>.map: Various information about the model (metrics, training indexes, features, ...).
  • models/<dataset>/<dataset>.<id>.instances: 10 instances per model from the test set, to be explained.
  • models/<dataset>/<dataset>.<id>.<percentage>.v2.references.json: Set RC of reference instances selected uniformly at random from the training set.
    The number of reference instances retained in RC varied to consider 5%, 10% or 20% of the instances (<percentage>).
• sources/pyxai-anchored/pyxai/examples/Converters/: The source code to convert each dataset:
  • Selection of numerical and categorical features,
  • One-hot encoding of categorical features,
  • Using the OneVsOne method to transform labels into binary classes (only for the datasets balance and arrowhead).
• logs: The outputs produced by the algorithm run in the experiments.
• proofs.pdf: The proofs of the propositions provided in the paper.
• datasets_information.pdf: Additional information about datasets (the number of features, numerical features, categorical features and Boolean features and instances, and then the link to the dataset website). These links allow to get the 14 datasets in their raw form.

Software

Setup

• For PyXAI, the installation procedure is the same as the one provided at https://www.cril.univ-artois.fr/pyxai/documentation/installation/github/
• Be sure to use a Linux OS and to use a version of Python >= to 3.8.
• cd sources/pyxai-anchored/
• Create and activate a new virtual environment:
• sudo apt-get install python3.10-venv
• python3 -m venv env
• source env/bin/activate
• Install dependencies:
• python3 -m pip install -e .
• Install encore (C++ program to compute k-anchored explanations):
• cd sources/encore/
• ./build.sh
• Unzip [Datasets] in the sources/pyxai-anchored/ijcai2024-anchored/ directory

How to use our program

• For a given dataset, the deterministic program sources/pyxai-anchored/ijcai2024-anchored/compute_anchored.py computes most anchored explanations for 100 instances picked up uniformly at random in the test set for each of reference instances percentage (5%, 10% or 20%).
• The instructions are executed from the sources/pyxai-anchored/ directory, so the python path must be configured in this way:
• cd sources/pyxai-anchored/pyxai/
• export PYTHONPATH="${PYTHONPATH}:${PWD}/.."
• The next command runs the algorithm on the set of 100 test instances per reference instances percentage (5%, 10% or 20%), with a timeout of 60 seconds per instance in order to produce 300 most enchored explanations using the theory defined in the paper.
  In this example, we have extracted the dataset in the sources/pyxai-anchored/datasets/ directory.
  
  cd sources/pyxai-anchored/
python3 ijcai2024-anchored/compute_anchored.py -model_directory=ijcai2024-anchored/datasets/models/arrowhead_1_vs_2/ -dataset=ijcai2024-anchored/datasets/datasets/ -time_limit=60 -type_references=normal
  
  
• Since our program is deterministic, the results produced for a given dataset must be the same ones as those reported in the logs folder.
• For each instance for which an explanation is derived, a trace of the computation, similar to the following one, is printed:

        Theory enabled: clauses: 5467 to 6062
        size reference_instances 0:  15
        size reference_instances 1:  31
        Check reason: OK
        id instance:  8
        prediction:  1
        id model:  2
        percentage references:  20
        time most_anchored_reason:  10.930509090423584
        n_anchors most_anchored_reason:  9
        size most_anchored_reason:  90
      

  This trace gives various information about the size of reference instances and the computation achieved, including the time needed to generate the most anchored explanation that is returned (on the example, 10.930509090423584) (the value 60 is used if a timeout has been reached), the current percentage of reference instances that is considered (on the example, 20), the maximal number of reference instances covered by the explanation (on the example, 9). A checker has also been implemented to verify that the explanation that is generated actually is k-anchored explanation (Check reason: OK). All the outputs produced by our algorithm for computing most anchored explanations are available in the .log files.