MachineLearningNotebooks/tutorials/get-started-day1/day1-part3-train-model.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/get-started-day1/day1-part3-train-model.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Tutorial: Train your first ML model (Part 3 of 4)\n",
        "\n",
        "---\n",
        "## Introduction\n",
        "In the [previous tutorial](day1-part2-hello-world.ipynb), you ran a trivial \"Hello world!\" script in the cloud using Azure Machine Learning's Python SDK. This time you take it a step further by submitting a script that will train a machine learning model. This example will help you understand how Azure Machine Learning eases consistent behavior between debugging on a compute instance or laptop development environment, and remote runs.\n",
        "\n",
        "Learning these concepts means that by the end of this session, you can:\n",
        "\n",
        "* Use Conda to define an Azure Machine Learning environment.\n",
        "* Train a model in the cloud.\n",
        "* Log metrics to Azure Machine Learning.\n",
        "\n",
        "This notebook follows the steps provided on the [Python (day 1) - train a model documentation page](https://aka.ms/day1aml).\n",
        "\n",
        "## Prerequisites\n",
        "\n",
        "- You have completed the following:\n",
        "  - [Setup on your compute cluster](day1-part1-setup.ipynb)\n",
        "  - [Tutorial: Hello World example](day1-part2-hello-world.md)\n",
        "- Familiarity with Python and Machine Learning concepts\n",
        "- If you are using a compute instance in Azure Machine Learning to run this notebook series, you are all set. Otherwise, please follow the [Configure a development environment for Azure Machine Learning](https://docs.microsoft.com/azure/machine-learning/how-to-configure-environment)\n",
        "---\n",
        "\n",
        "## Your machine learning code\n",
        "\n",
        "This tutorial shows you how to train a PyTorch model on the CIFAR 10 dataset using an Azure Machine Learning Cluster. In this case you will be using a CPU cluster, but this could equally be a GPU cluster. Whilst this tutorial uses PyTorch, the steps we show you apply to *any* machine learning code. \n",
        "\n",
        "In the `code/pytorch-cifar10-train` subdirectory you will see 2 files:\n",
        "\n",
        "1. [model.py](code/pytorch-cifar10-train/model.py) - this defines the neural network architecture\n",
        "1. [train.py](code/pytorch-cifar10-train/train.py) - This is the training script. This script downloads the CIFAR10 dataset using PyTorch `torchvision.dataset` APIs, sets up the network defined in\n",
        "`model.py`, and trains it for two epochs using standard SGD and cross-entropy loss.\n",
        "\n",
        "Note the code is based on [this introductory example from PyTorch](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html). "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Define the Python environment for your machine learning code\n",
        "\n",
        "For demonstration purposes, we're going to use a Conda environment but the steps for a pip virtual environment are almost identical. This environment has all the dependencies that your model and training script require. \n",
        "\n",
        "In the `configuration` directory there is a *conda dependencies* file called [pytorch-env.yml](configuration/pytorch-env.yml) that specifies the dependencies to run the python code. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Test in your development environment\n",
        "\n",
        "Test your script runs on either your compute instance or laptop using this environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "!python code/pytorch-cifar10-train/train.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**You should notice that the script has downloaded the data into a directory called `data`.**\n",
        "\n",
        "## Submit your machine learning code to Azure Machine Learning\n",
        "\n",
        "The difference to the control script below and the one used to submit \"hello world\" is that you adjust the environment to be set from the conda dependencies file you created earlier.\n",
        "\n",
        "> <span style=\"color:purple; font-weight:bold\">! NOTE <br>\n",
        "> The first time you run this script, Azure Machine Learning will build a new docker image from your PyTorch environment. The whole run could take 5-10 minutes to complete. You can see the docker build logs in the widget by selecting the `20_image_build_log.txt` in the log files dropdown. This image will be reused in future runs making them run much quicker.</span>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "remote run",
          "batchai",
          "configure run",
          "use notebook widget"
        ]
      },
      "outputs": [],
      "source": [
        "from azureml.core import Workspace, Experiment, Environment, ScriptRunConfig\n",
        "from azureml.widgets import RunDetails\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "experiment = Experiment(workspace=ws, name='day1-experiment-train')\n",
        "config = ScriptRunConfig(source_directory='code/pytorch-cifar10-train/', script='train.py', compute_target='cpu-cluster')\n",
        "\n",
        "env = Environment.from_conda_specification(name='pytorch-env', file_path='configuration/pytorch-env.yml')\n",
        "config.run_config.environment = env\n",
        "\n",
        "run = experiment.submit(config)\n",
        "\n",
        "RunDetails(run).show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Understand the control code\n",
        "\n",
        "Compared to the control script that submitted the \"hello world\" example, this control script introduces the following:\n",
        "\n",
        "| Code | Description\n",
        "| --- | --- |\n",
        "| `env = Environment.from_conda_specification( ...)` | Azure Machine Learning provides the concept of an `Environment` to represent a reproducible, <br>versioned Python environment for running experiments. Here you have created it from a yaml conda dependencies file.|\n",
        "| `config.run_config.environment = env` | adds the environment to the ScriptRunConfig. |\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**There are many ways to create AML environments, including [from a pip requirements.txt](https://docs.microsoft.com/python/api/azureml-core/azureml.core.environment.environment?view=azure-ml-py&preserve-view=true#from-pip-requirements-name--file-path-), or even [from an existing local Conda environment](https://docs.microsoft.com/python/api/azureml-core/azureml.core.environment.environment?view=azure-ml-py&preserve-view=true#from-existing-conda-environment-name--conda-environment-name-).**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Once your image is built, select `70_driver_log.txt` to see the output of your training script, which should look like:\n",
        "\n",
        "```txt\n",
        "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz\n",
        "...\n",
        "Files already downloaded and verified\n",
        "epoch=1, batch= 2000: loss 2.19\n",
        "...\n",
        "epoch=2, batch=12000: loss 1.27\n",
        "Finished Training\n",
        "```\n",
        "\n",
        "Environments can be registered to a workspace with `env.register(ws)`, allowing them to be easily shared, reused, and versioned. Environments make it easy to reproduce previous results and to collaborate with your team.\n",
        "\n",
        "Azure Machine Learning also maintains a collection of curated environments. These environments cover common ML scenarios and are backed by cached Docker images. Cached Docker images make the first remote run faster.\n",
        "\n",
        "In short, using registered environments can save you time! More details can be found on the [environments documentation](./how-to-use-environments.md)\n",
        "\n",
        "## Log training metrics\n",
        "\n",
        "Now that you have a model training in Azure Machine Learning, start tracking some performance metrics.\n",
        "The current training script prints metrics to the terminal. Azure Machine Learning provides a\n",
        "mechanism for logging metrics with more functionality. By adding a few lines of code, you gain the ability to visualize metrics in the studio and to compare metrics between multiple runs.\n",
        "\n",
        "### Machine learning code updates\n",
        "\n",
        "In the `code/pytorch-cifar10-train-with-logging` directory you will notice the [train.py](code/pytorch-cifar10-train-with-logging/train.py) script has been modified with two additional lines that will log the loss to the Azure Machine Learning Studio:\n",
        "\n",
        "```python\n",
        "# in train.py\n",
        "run = Run.get_context()\n",
        "...\n",
        "run.log('loss', loss)\n",
        "```\n",
        "\n",
        "Metrics in Azure Machine Learning are:\n",
        "\n",
        "- Organized by experiment and run so it's easy to keep track of and\n",
        "compare metrics.\n",
        "- Equipped with a UI so we can visualize training performance in the studio or in the notebook widget.\n",
        "- **Designed to scale** You can submit concurrent experiments and the Azure Machine Learning cluster will scale out (up to the maximum node count of the cluster) to run the experiments in parallel."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Update the Environment for your machine learning code\n",
        "\n",
        "The `train.py` script just took a new dependency on `azureml.core`. Therefore, the conda dependecies file [pytorch-aml-env](configuration/pytorch-aml-env.yml) reflects this change."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Submit your machine learning code to Azure Machine Learning\n",
        "Submit your code once more. This time the widget includes the metrics where you can now see live updates on the model training loss!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [
          "remote run",
          "batchai",
          "configure run",
          "use notebook widget",
          "get metrics"
        ]
      },
      "outputs": [],
      "source": [
        "from azureml.core import Workspace, Experiment, Environment, ScriptRunConfig\n",
        "from azureml.widgets import RunDetails\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "experiment = Experiment(workspace=ws, name='day1-experiment-train')\n",
        "config = ScriptRunConfig(source_directory='code/pytorch-cifar10-train-with-logging', script='train.py', compute_target='cpu-cluster')\n",
        "\n",
        "env = Environment.from_conda_specification(name='pytorch-aml-env', file_path='configuration/pytorch-aml-env.yml')\n",
        "config.run_config.environment = env\n",
        "\n",
        "run = experiment.submit(config)\n",
        "RunDetails(run).show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Next steps\n",
        "\n",
        "In this session, you upgraded from a basic \"Hello world!\" script to a more realistic\n",
        "training script that required a specific Python environment to run. You saw how\n",
        "to take a local Conda environment to the cloud with Azure Machine Learning Environments. Finally, you\n",
        "saw how in a few lines of code you can log metrics to Azure Machine Learning.\n",
        "\n",
        "In the next session, you'll see how to work with data in Azure Machine Learning by uploading the CIFAR10\n",
        "dataset to Azure.\n",
        "\n",
        "[Tutorial: Bring your own data](day1-part4-data.ipynb)\n"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "samkemp"
      }
    ],
    "celltoolbar": "Edit Metadata",
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    },
    "notice": "Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License."
  },
  "nbformat": 4,
  "nbformat_minor": 4
}