MachineLearningNotebooks/how-to-use-azureml/explain-model/azure-integration/gpu-explanation/train-explain-model-gpu-tree-explainer.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/automated-machine-learning/classification-text-dnn/auto-ml-classification-text-dnn.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Explain tree-based models on GPU using GPUTreeExplainer\n",
        "\n",
        "\n",
        "_**This notebook illustrates how to use shap's GPUTreeExplainer on an Azure GPU machine.**_\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "Problem: Train a tree-based model and explain the model on an Azure GPU machine using the GPUTreeExplainer.\n",
        "\n",
        "---\n",
        "\n",
        "## Table of Contents\n",
        "\n",
        "1. [Introduction](#Introduction)\n",
        "1. [Setup](#Setup)\n",
        "1. [Run model explainer locally at training time](#Explain)\n",
        "    1. Apply feature transformations\n",
        "    1. Train a binary classification model\n",
        "    1. Explain the model on raw features\n",
        "        1. Generate global explanations\n",
        "        1. Generate local explanations\n",
        "1. [Visualize explanations](#Visualize)\n",
        "1. [Deploy model and scoring explainer](#Deploy)\n",
        "1. [Next steps](#Next)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Introduction\n",
        "This notebook demonstrates how to use the GPUTreeExplainer on some simple datasets.  Like the TreeExplainer, the GPUTreeExplainer is specifically designed for tree-based machine learning models, but it is designed to accelerate the computations using NVIDIA GPUs.\n",
        "\n",
        "\n",
        "Make sure you have executed the [configuration](../../../configuration.ipynb) before running this notebook.\n",
        "\n",
        "Notebook synopsis:\n",
        "\n",
        "1. Creating an Experiment in an existing Workspace\n",
        "2. Configuration and remote run with a GPU machine"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import logging\n",
        "import os\n",
        "import shutil\n",
        "\n",
        "import pandas as pd\n",
        "\n",
        "import azureml.core\n",
        "from azureml.core.experiment import Experiment\n",
        "from azureml.core.workspace import Workspace\n",
        "from azureml.core.dataset import Dataset\n",
        "from azureml.core.compute import AmlCompute\n",
        "from azureml.core.compute import ComputeTarget\n",
        "from azureml.core.run import Run\n",
        "from azureml.core.model import Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This sample notebook may use features that are not available in previous versions of the Azure ML SDK."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print(\"This notebook was created using version 1.51.0 of the Azure ML SDK\")\n",
        "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "As part of the setup you have already created a <b>Workspace</b>. To run the script, you also need to create an <b>Experiment</b>. An Experiment corresponds to a prediction problem you are trying to solve, while a Run corresponds to a specific approach to the problem."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "ws = Workspace.from_config()\n",
        "\n",
        "# Choose an experiment name.\n",
        "experiment_name = 'gpu-tree-explainer'\n",
        "\n",
        "experiment = Experiment(ws, experiment_name)\n",
        "\n",
        "output = {}\n",
        "output['Subscription ID'] = ws.subscription_id\n",
        "output['Workspace Name'] = ws.name\n",
        "output['Resource Group'] = ws.resource_group\n",
        "output['Location'] = ws.location\n",
        "output['Experiment Name'] = experiment.name\n",
        "pd.set_option('display.max_colwidth', -1)\n",
        "outputDf = pd.DataFrame(data = output, index = [''])\n",
        "outputDf.T"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Create project directory\n",
        "\n",
        "Create a directory that will contain all the necessary code from your local machine that you will need access to on the remote resource. This includes the training script, and any additional files your training script depends on"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "import shutil\n",
        "\n",
        "project_folder = './azureml-shap-gpu-tree-explainer'\n",
        "os.makedirs(project_folder, exist_ok=True)\n",
        "shutil.copy('gpu_tree_explainer.py', project_folder)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Set up a compute cluster\n",
        "This section uses a user-provided compute cluster (named \"gpu-shap-cluster\" in this example). If a cluster with this name does not exist in the user's workspace, the below code will create a new cluster. You can choose the parameters of the cluster as mentioned in the comments."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.compute import ComputeTarget, AmlCompute\n",
        "from azureml.core.compute_target import ComputeTargetException\n",
        "\n",
        "num_nodes = 1\n",
        "\n",
        "# Choose a name for your cluster.\n",
        "amlcompute_cluster_name = \"gpu-shap-cluster\"\n",
        "\n",
        "# Verify that cluster does not exist already\n",
        "try:\n",
        "    compute_target = ComputeTarget(workspace=ws, name=amlcompute_cluster_name)\n",
        "    print('Found existing cluster, use it.')\n",
        "except ComputeTargetException:\n",
        "    compute_config = AmlCompute.provisioning_configuration(vm_size = \"STANDARD_NC6\",\n",
        "                                                           # To use GPUTreeExplainer, select a GPU such as \"STANDARD_NC6\" \n",
        "                                                           # or similar GPU option\n",
        "                                                           # available in your workspace\n",
        "                                                           max_nodes = num_nodes)\n",
        "    compute_target = ComputeTarget.create(ws, amlcompute_cluster_name, compute_config)\n",
        "\n",
        "compute_target.wait_for_completion(show_output=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Configure & Run"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.runconfig import RunConfiguration\n",
        "from azureml.core.conda_dependencies import CondaDependencies\n",
        "\n",
        "# Create a new RunConfig object\n",
        "run_config = RunConfiguration(framework=\"python\")\n",
        "\n",
        "# Set compute target to AmlCompute target created in previous step\n",
        "run_config.target = amlcompute_cluster_name\n",
        "\n",
        "from azureml.core import Environment\n",
        "\n",
        "environment_name = \"shapgpu\"\n",
        "env = Environment(environment_name)\n",
        "\n",
        "env.docker.enabled = True\n",
        "env.docker.base_image = None\n",
        "\n",
        "\n",
        "# Note: this is to pin the pandas and xgboost versions to be same as notebook.\n",
        "# In production scenario user would choose their dependencies\n",
        "import pkg_resources\n",
        "from distutils.version import LooseVersion\n",
        "available_packages = pkg_resources.working_set\n",
        "pandas_ver = None\n",
        "numpy_ver = None\n",
        "sklearn_ver = None\n",
        "for dist in list(available_packages):\n",
        "    if dist.key == 'pandas':\n",
        "       pandas_ver = dist.version\n",
        "    if dist.key == 'numpy':\n",
        "       if LooseVersion(dist.version) >= LooseVersion('1.20.0'):\n",
        "           numpy_ver = dist.version\n",
        "       else:\n",
        "           numpy_ver = '1.21.6'\n",
        "    if dist.key == 'scikit-learn':\n",
        "        sklearn_ver = dist.version\n",
        "pandas_dep = 'pandas'\n",
        "numpy_dep = 'numpy'\n",
        "sklearn_dep = 'scikit-learn'\n",
        "if pandas_ver:\n",
        "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
        "if numpy_ver:\n",
        "    numpy_dep = 'numpy=={}'.format(numpy_ver)\n",
        "if sklearn_ver:\n",
        "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
        "\n",
        "# Note: we build shap at commit 690245 for Tesla K80 GPUs\n",
        "env.docker.base_dockerfile = f\"\"\"\n",
        "FROM nvidia/cuda:10.2-devel-ubuntu18.04\n",
        "ENV PATH=\"/root/miniconda3/bin:${{PATH}}\"\n",
        "ARG PATH=\"/root/miniconda3/bin:${{PATH}}\"\n",
        "RUN apt-get update && \\\n",
        "apt-get install -y fuse && \\\n",
        "apt-get install -y build-essential && \\\n",
        "apt-get install -y python3-dev && \\\n",
        "apt-get install -y wget && \\\n",
        "apt-get install -y git && \\\n",
        "rm -rf /var/lib/apt/lists/* && \\\n",
        "wget \\\n",
        "https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \\\n",
        "mkdir /root/.conda && \\\n",
        "bash Miniconda3-latest-Linux-x86_64.sh -b && \\\n",
        "rm -f Miniconda3-latest-Linux-x86_64.sh && \\\n",
        "conda init bash && \\\n",
        ". ~/.bashrc && \\\n",
        "conda create -n shapgpu python=3.8 && \\\n",
        "conda activate shapgpu && \\\n",
        "apt-get install -y g++ && \\\n",
        "printenv && \\\n",
        "echo \"which nvcc: \" && \\\n",
        "which nvcc && \\\n",
        "pip install azureml-defaults && \\\n",
        "pip install azureml-telemetry && \\\n",
        "pip install azureml-interpret && \\\n",
        "pip install {pandas_dep} && \\\n",
        "cd /usr/local/src && \\\n",
        "git clone https://github.com/slundberg/shap.git --single-branch && \\\n",
        "cd shap && \\\n",
        "git reset --hard 690245c6ab043edf40cfce3d8438a62e29ab599f && \\\n",
        "mkdir build && \\\n",
        "python setup.py install --user && \\\n",
        "pip uninstall -y xgboost && \\\n",
        "conda install py-xgboost==1.3.3 && \\\n",
        "pip uninstall -y numpy && \\\n",
        "pip install {numpy_dep} && \\\n",
        "pip install {sklearn_dep} && \\\n",
        "pip install chardet \\\n",
        "\"\"\"\n",
        "\n",
        "env.python.user_managed_dependencies = True\n",
        "env.python.interpreter_path = '/root/miniconda3/envs/shapgpu/bin/python'\n",
        "\n",
        "from azureml.core import Run\n",
        "from azureml.core import ScriptRunConfig\n",
        "\n",
        "src = ScriptRunConfig(source_directory=project_folder, \n",
        "                      script='gpu_tree_explainer.py', \n",
        "                      compute_target=amlcompute_cluster_name,\n",
        "                      environment=env) \n",
        "run = experiment.submit(config=src)\n",
        "run"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Note: if you need to cancel a run, you can follow [these instructions](https://aka.ms/aml-docs-cancel-run)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "# Shows output of the run on stdout.\n",
        "run.wait_for_completion(show_output=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "run.get_metrics()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Download \n",
        "1. Download model explanation data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.interpret import ExplanationClient\n",
        "\n",
        "# Get model explanation data\n",
        "client = ExplanationClient.from_run(run)\n",
        "global_explanation = client.download_model_explanation()\n",
        "local_importance_values = global_explanation.local_importance_values\n",
        "expected_values = global_explanation.expected_values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Get the top k (e.g., 4) most important features with their importance values\n",
        "global_explanation_topk = client.download_model_explanation(top_k=4)\n",
        "global_importance_values = global_explanation_topk.get_ranked_global_values()\n",
        "global_importance_names = global_explanation_topk.get_ranked_global_names()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print('global importance values: {}'.format(global_importance_values))\n",
        "print('global importance names: {}'.format(global_importance_names))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "2. Download model file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Retrieve model for visualization and deployment\n",
        "from azureml.core.model import Model\n",
        "import joblib\n",
        "original_model = Model(ws, 'xgboost_with_gpu_tree_explainer')\n",
        "model_path = original_model.download(exist_ok=True)\n",
        "original_model = joblib.load(model_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "3. Download test dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Retrieve x_test for visualization\n",
        "x_test_path = './x_shap_adult_census.pkl'\n",
        "run.download_file('x_shap_adult_census.pkl', output_file_path=x_test_path)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "x_test = joblib.load('x_shap_adult_census.pkl')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Visualize\n",
        "Load the visualization dashboard"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from raiwidgets import ExplanationDashboard"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from interpret_community.common.model_wrapper import wrap_model\n",
        "from interpret_community.dataset.dataset_wrapper import DatasetWrapper\n",
        "# note we need to wrap the XGBoost model to output predictions and probabilities in the scikit-learn format\n",
        "class WrappedXGBoostModel(object):\n",
        "    \"\"\"A class for wrapping an XGBoost model to output integer predicted classes.\"\"\"\n",
        "\n",
        "    def __init__(self, model):\n",
        "        self.model = model\n",
        "\n",
        "    def predict(self, dataset):\n",
        "        return self.model.predict(dataset).astype(int)\n",
        "\n",
        "    def predict_proba(self, dataset):\n",
        "        return self.model.predict_proba(dataset)\n",
        "\n",
        "wrapped_model = WrappedXGBoostModel(wrap_model(original_model, DatasetWrapper(x_test), model_task='classification'))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "ExplanationDashboard(global_explanation, wrapped_model, dataset=x_test)"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "ilmat"
      }
    ],
    "kernelspec": {
      "display_name": "Python 3.8 - AzureML",
      "language": "python",
      "name": "python38-azureml"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.8"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}