{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/automated-machine-learning/production-deploy-to-aks-gpu/production-deploy-to-aks-gpu.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Deploying a web service to Azure Kubernetes Service (AKS)\n",
        "This notebook shows the steps for deploying a service: registering a model, creating an image, provisioning a cluster (one time action), and deploying a service to it. \n",
        "We then test and delete the service, image and model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import azureml.core\n",
        "print(azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Get workspace\n",
        "Load existing workspace from the config file info."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.workspace import Workspace\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Download the model\n",
        "\n",
        "Prior to registering the model, you should have a TensorFlow [Saved Model](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/saved_model/README.md) in the `resnet50` directory. This cell will download a [pretrained resnet50](http://download.tensorflow.org/models/official/20181001_resnet/savedmodels/resnet_v1_fp32_savedmodel_NCHW_jpg.tar.gz) and unpack it to that directory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "import requests\n",
        "import shutil\n",
        "import tarfile\n",
        "import tempfile\n",
        "\n",
        "from io import BytesIO\n",
        "\n",
        "model_url = \"http://download.tensorflow.org/models/official/20181001_resnet/savedmodels/resnet_v1_fp32_savedmodel_NCHW_jpg.tar.gz\"\n",
        "\n",
        "archive_prefix = \"./resnet_v1_fp32_savedmodel_NCHW_jpg/1538686758/\"\n",
        "target_folder = \"resnet50\"\n",
        "\n",
        "if not os.path.exists(target_folder):\n",
        "    response = requests.get(model_url)\n",
        "    archive = tarfile.open(fileobj=BytesIO(response.content))\n",
        "    with tempfile.TemporaryDirectory() as temp_folder:\n",
        "        archive.extractall(temp_folder)\n",
        "        shutil.copytree(os.path.join(temp_folder, archive_prefix), target_folder)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Register the model\n",
        "Register an existing trained model, add description and tags."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.model import Model\n",
        "\n",
        "model = Model.register(model_path=\"resnet50\", # This points to the local directory to upload.\n",
        "                       model_name=\"resnet50\", # This is the name the model is registered as.\n",
        "                       tags={'area': \"Image classification\", 'type': \"classification\"},\n",
        "                       description=\"Image classification trained on Imagenet Dataset\",\n",
        "                       workspace=ws)\n",
        "\n",
        "print(model.name, model.description, model.version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Provision the AKS Cluster\n",
        "This is a one time setup. You can reuse this cluster for multiple deployments after it has been created. If you delete the cluster or the resource group that contains it, then you would have to recreate it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.compute import ComputeTarget, AksCompute\n",
        "from azureml.core.compute_target import ComputeTargetException\n",
        "\n",
        "# Choose a name for your GPU cluster\n",
        "gpu_cluster_name = \"aks-gpu-cluster\"\n",
        "\n",
        "# Verify that cluster does not exist already\n",
        "try:\n",
        "    gpu_cluster = ComputeTarget(workspace=ws, name=gpu_cluster_name)\n",
        "    print(\"Found existing gpu cluster\")\n",
        "except ComputeTargetException:\n",
        "    print(\"Creating new gpu-cluster\")\n",
        "    \n",
        "    # Specify the configuration for the new cluster\n",
        "    compute_config = AksCompute.provisioning_configuration(cluster_purpose=AksCompute.ClusterPurpose.DEV_TEST,\n",
        "                                                           agent_count=1,\n",
        "                                                           vm_size=\"Standard_NV6\")\n",
        "    # Create the cluster with the specified name and configuration\n",
        "    gpu_cluster = ComputeTarget.create(ws, gpu_cluster_name, compute_config)\n",
        "\n",
        "    # Wait for the cluster to complete, show the output log\n",
        "    gpu_cluster.wait_for_completion(show_output=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Deploy the model as a web service to AKS\n",
        "\n",
        "First create a scoring script"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%writefile score.py\n",
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import json\n",
        "import os\n",
        "from azureml.contrib.services.aml_request import AMLRequest, rawhttp\n",
        "from azureml.contrib.services.aml_response import AMLResponse\n",
        "\n",
        "def init():\n",
        "    global session\n",
        "    global input_name\n",
        "    global output_name\n",
        "    \n",
        "    session = tf.Session()\n",
        "\n",
        "    # AZUREML_MODEL_DIR is an environment variable created during deployment.\n",
        "    # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)\n",
        "    # For multiple models, it points to the folder containing all deployed models (./azureml-models)\n",
        "    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'resnet50')\n",
        "    model = tf.saved_model.loader.load(session, ['serve'], model_path)\n",
        "    if len(model.signature_def['serving_default'].inputs) > 1:\n",
        "        raise ValueError(\"This score.py only supports one input\")\n",
        "    input_name = [tensor.name for tensor in model.signature_def['serving_default'].inputs.values()][0]\n",
        "    output_name = [tensor.name for tensor in model.signature_def['serving_default'].outputs.values()]\n",
        "    \n",
        "\n",
        "@rawhttp\n",
        "def run(request):\n",
        "    if request.method == 'POST':\n",
        "        reqBody = request.get_data(False)\n",
        "        resp = score(reqBody)\n",
        "        return AMLResponse(resp, 200)\n",
        "    if request.method == 'GET':\n",
        "        respBody = str.encode(\"GET is not supported\")\n",
        "        return AMLResponse(respBody, 405)\n",
        "    return AMLResponse(\"bad request\", 500)\n",
        "\n",
        "def score(data):\n",
        "    result = session.run(output_name, {input_name: [data]})\n",
        "    return json.dumps(result[1].tolist())\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    init()\n",
        "    with open(\"test_image.jpg\", 'rb') as f:\n",
        "        content = f.read()\n",
        "        print(score(content))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now create the deployment configuration objects and deploy the model as a webservice."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Set the web service configuration (using default here)\n",
        "from azureml.core.model import InferenceConfig\n",
        "from azureml.core.webservice import AksWebservice\n",
        "from azureml.core.conda_dependencies import CondaDependencies\n",
        "from azureml.core.environment import Environment, DEFAULT_GPU_IMAGE\n",
        "\n",
        "env = Environment('deploytocloudenv')\n",
        "# Please see [Azure ML Containers repository](https://github.com/Azure/AzureML-Containers#featured-tags)\n",
        "# for open-sourced GPU base images.\n",
        "env.docker.base_image = DEFAULT_GPU_IMAGE\n",
        "env.python.conda_dependencies = CondaDependencies.create(conda_packages=['tensorflow-gpu==1.12.0','numpy'],\n",
        "                                 pip_packages=['azureml-contrib-services', 'azureml-defaults'])\n",
        "\n",
        "inference_config = InferenceConfig(entry_script=\"score.py\", environment=env)\n",
        "aks_config = AksWebservice.deploy_configuration()\n",
        "\n",
        "# # Enable token auth and disable (key) auth on the webservice\n",
        "# aks_config = AksWebservice.deploy_configuration(token_auth_enabled=True, auth_enabled=False)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "aks_service_name ='gpu-rn50'\n",
        "\n",
        "aks_service = Model.deploy(workspace=ws,\n",
        "                           name=aks_service_name,\n",
        "                           models=[model],\n",
        "                           inference_config=inference_config,\n",
        "                           deployment_config=aks_config,\n",
        "                           deployment_target=gpu_cluster)\n",
        "\n",
        "aks_service.wait_for_deployment(show_output = True)\n",
        "print(aks_service.state)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Test the web service\n",
        "We test the web sevice by passing the test images content."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "import requests\n",
        "\n",
        "# if (key) auth is enabled, fetch keys and include in the request\n",
        "key1, key2 = aks_service.get_keys()\n",
        "\n",
        "headers = {'Content-Type':'application/json', 'Authorization': 'Bearer ' + key1}\n",
        "\n",
        "# # if token auth is enabled, fetch token and include in the request\n",
        "# access_token, fetch_after = aks_service.get_token()\n",
        "# headers = {'Content-Type':'application/json', 'Authorization': 'Bearer ' + access_token}\n",
        "\n",
        "test_sample = open('snowleopardgaze.jpg', 'rb').read()\n",
        "resp = requests.post(aks_service.scoring_uri, test_sample, headers=headers)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Clean up\n",
        "Delete the service, image, model and compute target"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "aks_service.delete()\n",
        "model.delete()\n",
        "gpu_cluster.delete()\n"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "vaidyas"
      }
    ],
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}