MachineLearningNotebooks/how-to-use-azureml/deployment/enable-app-insights-in-production-service/enable-app-insights-in-production-service.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Enabling App Insights for Services in Production\n",
        "With this notebook, you can learn how to enable App Insights for standard service monitoring, plus, we provide examples for doing custom logging within a scoring files in a model.\n",
        "\n",
        "\n",
        "## What does Application Insights monitor?\n",
        "It monitors request rates, response times, failure rates, etc. For more information visit [App Insights docs.](https://docs.microsoft.com/en-us/azure/application-insights/app-insights-overview)\n",
        "\n",
        "\n",
        "## What is different compared to standard production deployment process?\n",
        "If you want to enable generic App Insights for a service run:\n",
        "```python\n",
        "aks_service= Webservice(ws, \"aks-w-dc2\")\n",
        "aks_service.update(enable_app_insights=True)```\n",
        "Where \"aks-w-dc2\" is your service name. You can also do this from the Azure Portal under your Workspace--> deployments--> Select deployment--> Edit--> Advanced Settings--> Select \"Enable AppInsights diagnostics\"\n",
        "\n",
        "If you want to log custom traces, you will follow the standard deplyment process for AKS and you will:\n",
        "1. Update scoring file.\n",
        "2. Update aks configuration.\n",
        "3. Deploy the model with this new configuration. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/deployment/enable-app-insights-in-production-service/enable-app-insights-in-production-service.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 1. Import your dependencies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import azureml.core\n",
        "import json\n",
        "\n",
        "from azureml.core import Workspace\n",
        "from azureml.core.compute import AksCompute, ComputeTarget\n",
        "from azureml.core.webservice import AksWebservice\n",
        "\n",
        "print(azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 2. Set up your configuration and create a workspace\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "ws = Workspace.from_config()\n",
        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 3. Register Model\n",
        "Register an existing trained model, add descirption and tags."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Model\n",
        "\n",
        "model = Model.register(model_path=\"sklearn_regression_model.pkl\", # This points to a local file.\n",
        "                       model_name=\"sklearn_regression_model.pkl\", # This is the name the model is registered as.\n",
        "                       tags={'area': \"diabetes\", 'type': \"regression\"},\n",
        "                       description=\"Ridge regression model to predict diabetes\",\n",
        "                       workspace=ws)\n",
        "\n",
        "print(model.name, model.description, model.version)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 4. *Update your scoring file with custom print statements*\n",
        "Here is an example:\n",
        "### a. In your init function add:\n",
        "```python\n",
        "print (\"model initialized\" + time.strftime(\"%H:%M:%S\"))```\n",
        "\n",
        "### b. In your run function add:\n",
        "```python\n",
        "print (\"Prediction created\" + time.strftime(\"%H:%M:%S\"))```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%writefile score.py\n",
        "import os\n",
        "import pickle\n",
        "import json\n",
        "import numpy\n",
        "from sklearn.externals import joblib\n",
        "from sklearn.linear_model import Ridge\n",
        "import time\n",
        "\n",
        "def init():\n",
        "    global model\n",
        "    #Print statement for appinsights custom traces:\n",
        "    print (\"model initialized\" + time.strftime(\"%H:%M:%S\"))\n",
        "\n",
        "    # AZUREML_MODEL_DIR is an environment variable created during deployment.\n",
        "    # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)\n",
        "    # For multiple models, it points to the folder containing all deployed models (./azureml-models)\n",
        "    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_regression_model.pkl')\n",
        "\n",
        "    # deserialize the model file back into a sklearn model\n",
        "    model = joblib.load(model_path)\n",
        "\n",
        "\n",
        "# note you can pass in multiple rows for scoring\n",
        "def run(raw_data):\n",
        "    try:\n",
        "        data = json.loads(raw_data)['data']\n",
        "        data = numpy.array(data)\n",
        "        result = model.predict(data)\n",
        "        print (\"Prediction created\" + time.strftime(\"%H:%M:%S\"))\n",
        "        # you can return any datatype as long as it is JSON-serializable\n",
        "        return result.tolist()\n",
        "    except Exception as e:\n",
        "        error = str(e)\n",
        "        print (error + time.strftime(\"%H:%M:%S\"))\n",
        "        return error"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 5. *Create myenv.yml file*\n",
        "Please note that you must indicate azureml-defaults with verion >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.conda_dependencies import CondaDependencies\n",
        "\n",
        "myenv = CondaDependencies.create(conda_packages=['numpy==1.19.5','scikit-learn==0.22.1'],\n",
        "                                 pip_packages=['azureml-defaults'])\n",
        "\n",
        "with open(\"myenv.yml\",\"w\") as f:\n",
        "    f.write(myenv.serialize_to_string())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 6. Create Inference Configuration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.environment import Environment\n",
        "from azureml.core.model import InferenceConfig\n",
        "\n",
        "myenv = Environment.from_conda_specification(name=\"myenv\", file_path=\"myenv.yml\")\n",
        "inference_config = InferenceConfig(entry_script=\"score.py\", environment=myenv)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Deploy to ACI (Optional)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.webservice import AciWebservice\n",
        "\n",
        "aci_deployment_config = AciWebservice.deploy_configuration(cpu_cores=1,\n",
        "                                                           memory_gb=1,\n",
        "                                                           tags={'area': \"diabetes\", 'type': \"regression\"},\n",
        "                                                           description=\"Predict diabetes using regression model\",\n",
        "                                                           enable_app_insights=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "aci_service_name = \"aci-service-appinsights\"\n",
        "\n",
        "aci_service = Model.deploy(ws, aci_service_name, [model], inference_config, aci_deployment_config, overwrite=True)\n",
        "aci_service.wait_for_deployment(show_output=True)\n",
        "\n",
        "print(aci_service.state)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "if aci_service.state == \"Healthy\":\n",
        "    test_sample = json.dumps({\n",
        "        \"data\": [\n",
        "            [1,28,13,45,54,6,57,8,8,10],\n",
        "            [101,9,8,37,6,45,4,3,2,41]\n",
        "        ]\n",
        "    })\n",
        "\n",
        "    prediction = aci_service.run(test_sample)\n",
        "\n",
        "    print(prediction)\n",
        "else:\n",
        "    raise ValueError(\"Service deployment isn't healthy, can't call the service. Error: \", aci_service.error)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 7. Deploy to AKS service"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Create AKS compute if you haven't done so.\n",
        "\n",
        "> Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core.compute import ComputeTarget, AksCompute\n",
        "from azureml.core.compute_target import ComputeTargetException\n",
        "\n",
        "aks_name = \"my-aks-insights\"\n",
        "\n",
        "creating_compute = False\n",
        "try:\n",
        "    aks_target = ComputeTarget(ws, aks_name)\n",
        "    print(\"Using existing AKS compute target {}.\".format(aks_name))\n",
        "except ComputeTargetException:\n",
        "    print(\"Creating a new AKS compute target {}.\".format(aks_name))\n",
        "\n",
        "    # Use the default configuration (can also provide parameters to customize).\n",
        "    prov_config = AksCompute.provisioning_configuration()\n",
        "    aks_target = ComputeTarget.create(workspace=ws,\n",
        "                                      name=aks_name,\n",
        "                                      provisioning_configuration=prov_config)\n",
        "    creating_compute = True"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "if creating_compute and aks_target.provisioning_state != \"Succeeded\":\n",
        "    aks_target.wait_for_completion(show_output=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "print(aks_target.provisioning_state)\n",
        "print(aks_target.provisioning_errors)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "If you already have a cluster you can attach the service to it:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "```python\n",
        "%%time\n",
        "resource_id = '/subscriptions/<subscriptionid>/resourcegroups/<resourcegroupname>/providers/Microsoft.ContainerService/managedClusters/<aksservername>'\n",
        "create_name= 'myaks4'\n",
        "attach_config = AksCompute.attach_configuration(resource_id=resource_id)\n",
        "aks_target = ComputeTarget.attach(workspace=ws,\n",
        "                                  name=create_name,\n",
        "                                  attach_configuration=attach_config)\n",
        "## Wait for the operation to complete\n",
        "aks_target.wait_for_provisioning(True)```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### a. *Activate App Insights through updating AKS Webservice configuration*\n",
        "In order to enable App Insights in your service you will need to update your AKS configuration file:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Set the web service configuration.\n",
        "aks_deployment_config = AksWebservice.deploy_configuration(enable_app_insights=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### b. Deploy your service"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "if aks_target.provisioning_state == \"Succeeded\":\n",
        "    aks_service_name = \"aks-service-appinsights\"\n",
        "    aks_service = Model.deploy(ws,\n",
        "                               aks_service_name,\n",
        "                               [model],\n",
        "                               inference_config,\n",
        "                               aks_deployment_config,\n",
        "                               deployment_target=aks_target,\n",
        "                               overwrite=True)\n",
        "    aks_service.wait_for_deployment(show_output=True)\n",
        "    print(aks_service.state)\n",
        "else:\n",
        "    raise ValueError(\"AKS cluster provisioning failed. Error: \", aks_target.provisioning_errors)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 8. Test your service "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "\n",
        "if aks_service.state == \"Healthy\":\n",
        "    test_sample = json.dumps({\n",
        "        \"data\": [\n",
        "            [1,28,13,45,54,6,57,8,8,10],\n",
        "            [101,9,8,37,6,45,4,3,2,41]\n",
        "        ]\n",
        "    })\n",
        "\n",
        "    prediction = aks_service.run(input_data=test_sample)\n",
        "    print(prediction)\n",
        "else:\n",
        "    raise ValueError(\"Service deployment isn't healthy, can't call the service. Error: \", aks_service.error)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 9. See your service telemetry in App Insights\n",
        "1. Go to the [Azure Portal](https://portal.azure.com/)\n",
        "2. All resources--> Select the subscription/resource group where you created your Workspace--> Select the App Insights type\n",
        "3. Click on the AppInsights resource. You'll see a highlevel dashboard with information on Requests, Server response time and availability.\n",
        "4. Click on the top banner \"Analytics\"\n",
        "5. In the \"Schema\" section select \"traces\" and run your query.\n",
        "6. Voila! All your custom traces should be there."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Disable App Insights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "aks_service.update(enable_app_insights=False)\n",
        "aks_service.wait_for_deployment(show_output=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Clean up"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "%%time\n",
        "aks_service.delete()\n",
        "aci_service.delete()\n",
        "model.delete()\n",
        "if creating_compute:\n",
        "    aks_target.delete()"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "gopalv"
      }
    ],
    "kernelspec": {
      "display_name": "Python 3.8 - AzureML",
      "language": "python",
      "name": "python38-azureml"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}