{ "cells": [ { "cell_type": "markdown", "id": "4800aa8d", "metadata": {}, "source": [ "# Rectification for Decision Trees" ] }, { "cell_type": "markdown", "id": "87af483d", "metadata": {}, "source": [ "Further information on the rectification method for Decision Trees is available in the paper [Rectifying Binary Classifiers](https://hal.science/hal-04236309/document)." ] }, { "cell_type": "markdown", "id": "1dc05e33", "metadata": {}, "source": [ "## Example from a Hand-Crafted Tree" ] }, { "cell_type": "markdown", "id": "37a0365c", "metadata": {}, "source": [ "To illustrate this, we take an example of a credit scoring scenario. \n", "\n", "Each customer is characterized by:\n", "- an annual income $I$ (in k\\\\$),\n", "- the fact of having already reimbursed a previous loan ($R$),\n", "- and, whether or not, the customer has a permanent position ($PP$). \n", "\n", "A decision tree T representing the model is described in the following figure. The Boolean conditions used in $T$ are $I > 30$, $I > 20$, $R$, and $PP$. \n", "\n", "\"DTRectify1\"\n", "\n", "We start by building the decision tree: " ] }, { "cell_type": "code", "execution_count": 1, "id": "db883698", "metadata": {}, "outputs": [], "source": [ "from pyxai import Builder, Explaining\n", "\n", "node_L_1 = Builder.DecisionNode(3, operator=Builder.EQ, threshold=1, left=0, right=1)\n", "node_L_2 = Builder.DecisionNode(1, operator=Builder.GT, threshold=20, left=0, right=node_L_1)\n", "\n", "node_R_1 = Builder.DecisionNode(3, operator=Builder.EQ, threshold=1, left=0, right=1)\n", "node_R_2 = Builder.DecisionNode(2, operator=Builder.EQ, threshold=1, left=node_R_1, right=1)\n", "\n", "root = Builder.DecisionNode(1, operator=Builder.GT, threshold=30, left=node_L_2, right=node_R_2)\n", "tree = Builder.DecisionTree(3, root, feature_names=[\"I\", \"PP\", \"R\"])" ] }, { "cell_type": "markdown", "id": "aca44977", "metadata": {}, "source": [ "Consider the instance $x = (I = 25, R = 1, PP = 1)$ corresponding to a customer applying for a loan.\n", "We initialize the explainer with this instance and the associated theory (see the [Theories](/documentation/explainer/theories/) page for more information). " ] }, { "cell_type": "code", "execution_count": 2, "id": "30b36230", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "feature_names: ['I', 'PP', 'R']\n", "--------- Theory Feature Types -----------\n", "Before the one-hot encoding of categorical features:\n", "Numerical features: 1\n", "Categorical features: 0\n", "Binary features: 2\n", "Number of features: 3\n", "Characteristics of categorical features: {}\n", "\n", "Number of used features in the model (before the encoding of categorical features): 3\n", "Number of used features in the model (after the encoding of categorical features): 3\n", "----------------------------------------------\n", "binary representation: (-1, 2, 3, 4)\n", "target_prediction: 1\n", "to_features: ['I <= 30', 'I > 20', 'PP == 1', 'R == 1']\n" ] } ], "source": [ "loan_types = {\n", " \"numerical\": [\"I\"],\n", " \"binary\": [\"PP\", \"R\"],\n", "}\n", "\n", "explainer = Explaining.initialize(tree, instance=(25, 1, 1), features_type=loan_types)\n", "print(\"binary representation: \", explainer.binary_representation)\n", "print(\"target_prediction:\", explainer.target_prediction)\n", "print(\"to_features:\", explainer.to_features(explainer.binary_representation, eliminate_redundant_features=False))" ] }, { "cell_type": "markdown", "id": "452f1209", "metadata": {}, "source": [ "The user (a bank employee) disagrees with this prediction (the loan acceptance). For him/her, the following classification rule must be obeyed: whenever the annual income of the client is lower than 30, the demand should be rejected. To do this rectification, we use the ```rectify()``` method of the ```explainer``` object. More information about this method are available on the [Rectification](/documentation/rectification/) page." ] }, { "cell_type": "code", "execution_count": 3, "id": "f2f3bc71", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Rectify - Number of nodes - Initial (c++): 11\n", "Rectify - Number of nodes - After rectification (c++): 11\n", "Rectify - Number of nodes - After simplification with the theory (c++): 11\n", "Rectify - Number of nodes - After elimination of redundant nodes (c++): 7\n", "Rectify - Number of nodes - Final (c++): 7\n", "Rectification time: 0.00039471000000013134\n", "--------------\n", "target_prediction: 0\n" ] } ], "source": [ "rectified_model = explainer.rectify(conditions=(-1, ), label=0) \n", "# Keep in mind that the condition (-1, ) means that 'I <= 30'.\n", "\n", "print(\"target_prediction:\", explainer.target_prediction)" ] }, { "cell_type": "markdown", "id": "779a7def", "metadata": {}, "source": [ "Here is the model without any simplification: \n", "\n", "\"DTRectify2\"\n", "\n", "Here is the model once simplified:\n", "\n", "\"DTRectify3\"\n", "\n" ] }, { "cell_type": "markdown", "id": "44c1f638", "metadata": {}, "source": [ "We can check that the instance is now correctly classified:" ] }, { "cell_type": "code", "execution_count": 5, "id": "99697598", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "target_prediction: 0\n" ] } ], "source": [ "print(\"target_prediction:\", rectified_model.predict_instance((25, 1, 1)))" ] }, { "cell_type": "markdown", "id": "aa82967b", "metadata": {}, "source": [ "## Example from a Real Dataset" ] }, { "cell_type": "markdown", "id": "29048b47", "metadata": {}, "source": "For this example, we take the compas.csv dataset. We create a model using the hold-out approach (by default, the test size is set to 30%) and select a misclassified instance. " }, { "cell_type": "code", "execution_count": 4, "id": "83b88fc4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-------------- Information ---------------\n", "Problem type: classification\n", "Instances type: tabular\n", "Labels type: classes\n", "\n", "Dataset path: ../dataset/compas.csv\n", "nFeatures (nAttributes, with the labels): 11\n", "nInstances (nObservations): 6172\n", "nLabels: 2\n", "--------------- Model creation, fitting and evaluation ---------------\n", "Splitting method: hold-out\n", "Problem type: classification\n", "Models type: decision-tree\n", "model_parameters: {}\n", "--------- Evaluation Information ---------\n", "For the evaluation number 0:\n", "Metrics:\n", " sklearn_confusion_matrix: [[663, 189], [342, 349]]\n", " precision: 64.86988847583643\n", " recall: 50.50651230101303\n", " f1_score: 56.794141578519124\n", " specificity: 77.8169014084507\n", " true_positive: 349\n", " true_negative: 663\n", " false_positive: 189\n", " false_negative: 342\n", " accuracy: 65.58651976668827\n", "Number of Training instances: 4629\n", "Number of Testing instances: 1543\n", "\n", "--------------- Explainer ----------------\n", "For the split number 0:\n", "**Decision Tree Model**\n", "nFeatures: 11\n", "nNodes: 556\n", "nVariables: 45\n", "\n", "--------------- Instances ----------------\n", "Number of instances selected: 1\n", "----------------------------------------------\n", "Misdemeanor 0\n", "Number_of_Priors 1\n", "score_factor 1\n", "Age_Above_FourtyFive 0\n", "Age_Below_TwentyFive 1\n", "African_American 0\n", "Asian 0\n", "Hispanic 0\n", "Native_American 0\n", "Other 0\n", "Female 0\n", "Name: 3367, dtype: int64\n" ] } ], "source": [ "from pyxai import Learning, Explaining\n", "\n", "learner = Learning.Scikitlearn(\"../dataset/compas.csv\", problem_type='classification')\n", "model = learner.evaluate(splitting_method=Learning.HOLD_OUT, model_type=Learning.DT, splitting_parameters={'random_state':0})\n", "\n", "dict_information = learner.get_instances(model, n=1, indexes=Learning.TEST, is_correct=False, details=True, seed=2)\n", "\n", "instance = dict_information[\"instance\"]\n", "print(instance)\n", "label = dict_information[\"label\"]\n", "prediction = dict_information[\"prediction\"]\n", "\n" ] }, { "cell_type": "markdown", "id": "01198b4d", "metadata": {}, "source": [ "We activate the explainer with the associated theory and the selected instance: " ] }, { "cell_type": "code", "execution_count": 5, "id": "4a9ca637", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "feature_names: ['Misdemeanor', 'Number_of_Priors', 'score_factor', 'Age_Above_FourtyFive', 'Age_Below_TwentyFive', 'African_American', 'Asian', 'Hispanic', 'Native_American', 'Other', 'Female']\n", "--------- Theory Feature Types -----------\n", "Before the one-hot encoding of categorical features:\n", "Numerical features: 1\n", "Categorical features: 2\n", "Binary features: 3\n", "Number of features: 6\n", "Characteristics of categorical features: {'African_American': ['{African_American,Asian,Hispanic,Native_American,Other}', 'African_American', ['African_American', 'Asian', 'Hispanic', 'Native_American', 'Other']], 'Asian': ['{African_American,Asian,Hispanic,Native_American,Other}', 'Asian', ['African_American', 'Asian', 'Hispanic', 'Native_American', 'Other']], 'Hispanic': ['{African_American,Asian,Hispanic,Native_American,Other}', 'Hispanic', ['African_American', 'Asian', 'Hispanic', 'Native_American', 'Other']], 'Native_American': ['{African_American,Asian,Hispanic,Native_American,Other}', 'Native_American', ['African_American', 'Asian', 'Hispanic', 'Native_American', 'Other']], 'Other': ['{African_American,Asian,Hispanic,Native_American,Other}', 'Other', ['African_American', 'Asian', 'Hispanic', 'Native_American', 'Other']], 'Age_Above_FourtyFive': ['Age', 'Above_FourtyFive', ['Above_FourtyFive', 'Below_TwentyFive']], 'Age_Below_TwentyFive': ['Age', 'Below_TwentyFive', ['Above_FourtyFive', 'Below_TwentyFive']]}\n", "\n", "Number of used features in the model (before the encoding of categorical features): 6\n", "Number of used features in the model (after the encoding of categorical features): 11\n", "----------------------------------------------\n" ] } ], "source": [ "compas_types = {\n", " \"numerical\": [\"Number_of_Priors\"],\n", " \"binary\": [\"Misdemeanor\", \"score_factor\", \"Female\"],\n", " \"categorical\": {\"{African_American,Asian,Hispanic,Native_American,Other}\": [\"African_American\", \"Asian\", \"Hispanic\", \"Native_American\", \"Other\"],\n", " \"Age*\": [\"Above_FourtyFive\", \"Below_TwentyFive\"]}\n", "}\n", "\n", "\n", "explainer = Explaining.initialize(model, instance=instance, features_type=compas_types)" ] }, { "cell_type": "markdown", "id": "d28ea08d", "metadata": {}, "source": [ "We compute why the model gives the prediction one for this instance:" ] }, { "cell_type": "code", "execution_count": 6, "id": "b72f0e44", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "current prediction: 1\n", "explanation: (1, 4, 5, -10, -12, -13)\n" ] } ], "source": [ "print(\"current prediction: \", explainer.target_prediction)\n", "reason = explainer.sufficient_reason(n=1)\n", "print(\"explanation:\", reason)\n", "#print(\"to_features:\", explainer.to_features(reason))" ] }, { "cell_type": "markdown", "id": "39f6fe97", "metadata": {}, "source": [ "Suppose that the user knows that every instance covered by the explanation (-1, -2, -3, -4) should be classified as a negative instance. The model must be rectified by the corresponding classification rule.\n", "Once the model has been corrected, the instance is classified as expected by the user:" ] }, { "cell_type": "code", "execution_count": 7, "id": "7e4c567a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Rectify - Number of nodes - Initial (c++): 1113\n", "Rectify - Number of nodes - After rectification (c++): 1113\n", "Rectify - Number of nodes - After simplification with the theory (c++): 1113\n", "Rectify - Number of nodes - After elimination of redundant nodes (c++): 665\n", "Rectify - Number of nodes - Final (c++): 665\n", "Rectification time: 0.008641170999998948\n", "--------------\n", "new prediction: 0\n" ] } ], "source": [ "model = explainer.rectify(conditions=reason, label=0) # We want to change the prediction\n", "print(\"new prediction:\", explainer.target_prediction)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.7" } }, "nbformat": 4, "nbformat_minor": 5 }