MachineLearningNotebooks/01.getting-started/03.train-on-aci/03.train-on-aci.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# 03. Train on Azure Container Instance\n",
        "\n",
        "* Create Workspace\n",
        "* Create `train.py` in the project folder.\n",
        "* Configure an ACI (Azure Container Instance) run\n",
        "* Execute in ACI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Prerequisites\n",
        "Make sure you go through the [00. Installation and Configuration](00.configuration.ipynb) Notebook first if you haven't."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Check core SDK version number\n",
        "import azureml.core\n",
        "\n",
        "print(\"SDK version:\", azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Initialize Workspace\n",
        "\n",
        "Initialize a workspace object from persisted configuration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "create workspace"
        ]
      },
      "outputs": [],
      "source": [
        "from azureml.core import Workspace\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create An Experiment\n",
        "\n",
        "**Experiment** is a logical container in an Azure ML Workspace. It hosts run records which can include run metrics and output artifacts from your experiments."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Experiment\n",
        "experiment_name = 'train-on-aci'\n",
        "experiment = Experiment(workspace = ws, name = experiment_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Remote execution on ACI\n",
        "\n",
        "The training script `train.py` is already created for you. Let's have a look."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "with open('./train.py', 'r') as f:\n",
        "    print(f.read())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Configure for using ACI\n",
        "Linux-based ACI is available in `West US`, `East US`, `West Europe`, `North Europe`, `West US 2`, `Southeast Asia`, `Australia East`, `East US 2`, and `Central US` regions. See details [here](https://docs.microsoft.com/en-us/azure/container-instances/container-instances-quotas#region-availability)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "configure run"
        ]
      },
      "outputs": [],
      "source": [
        "from azureml.core.runconfig import RunConfiguration\n",
        "from azureml.core.conda_dependencies import CondaDependencies\n",
        "\n",
        "# create a new runconfig object\n",
        "run_config = RunConfiguration()\n",
        "\n",
        "# signal that you want to use ACI to execute script.\n",
        "run_config.target = \"containerinstance\"\n",
        "\n",
        "# ACI container group is only supported in certain regions, which can be different than the region the Workspace is in.\n",
        "run_config.container_instance.region = 'eastus2'\n",
        "\n",
        "# set the ACI CPU and Memory \n",
        "run_config.container_instance.cpu_cores = 1\n",
        "run_config.container_instance.memory_gb = 2\n",
        "\n",
        "# enable Docker \n",
        "run_config.environment.docker.enabled = True\n",
        "\n",
        "# set Docker base image to the default CPU-based image\n",
        "run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE\n",
        "\n",
        "# use conda_dependencies.yml to create a conda environment in the Docker image for execution\n",
        "run_config.environment.python.user_managed_dependencies = False\n",
        "\n",
        "# auto-prepare the Docker image when used for execution (if it is not already prepared)\n",
        "run_config.auto_prepare_environment = True\n",
        "\n",
        "# specify CondaDependencies obj\n",
        "run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Submit the Experiment\n",
        "Finally, run the training job on the ACI"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "remote run",
          "aci"
        ]
      },
      "outputs": [],
      "source": [
        "%%time \n",
        "from azureml.core.script_run_config import ScriptRunConfig\n",
        "\n",
        "script_run_config = ScriptRunConfig(source_directory='./',\n",
        "                                    script='train.py',\n",
        "                                    run_config=run_config)\n",
        "\n",
        "run = experiment.submit(script_run_config)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "query history"
        ]
      },
      "outputs": [],
      "source": [
        "# Show run details\n",
        "run"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "remote run",
          "aci"
        ]
      },
      "outputs": [],
      "source": [
        "%%time\n",
        "# Shows output of the run on stdout.\n",
        "run.wait_for_completion(show_output=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "get metrics"
        ]
      },
      "outputs": [],
      "source": [
        "# get all metris logged in the run\n",
        "run.get_metrics()\n",
        "metrics = run.get_metrics()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "print('When alpha is {1:0.2f}, we have min MSE {0:0.2f}.'.format(\n",
        "    min(metrics['mse']), \n",
        "    metrics['alpha'][np.argmin(metrics['mse'])]\n",
        "))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# show all the files stored within the run record\n",
        "run.get_file_names()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now you can take a model produced here, register it and then deploy as a web service."
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "roastala"
      }
    ],
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}