MachineLearningNotebooks/training/08.export-run-history-to-tensorboard/08.export-run-history-to-tensorboard.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
        "\n",
        "Licensed under the MIT License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# 41. Export Run History as Tensorboard logs\n",
        "\n",
        "1. Run some training and log some metrics into Run History\n",
        "2. Export the run history to some directory as Tensorboard logs\n",
        "3. Launch a local Tensorboard to view the run history"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Prerequisites\n",
        "Make sure you go through the [00. Installation and Configuration](00.configuration.ipynb) Notebook first if you haven't."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Check core SDK version number\n",
        "import azureml.core\n",
        "\n",
        "print(\"SDK version:\", azureml.core.VERSION)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Initialize Workspace\n",
        "\n",
        "Initialize a workspace object from persisted configuration."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.core import Workspace, Run, Experiment\n",
        "\n",
        "\n",
        "ws = Workspace.from_config()\n",
        "print('Workspace name: ' + ws.name, \n",
        "      'Azure region: ' + ws.location, \n",
        "      'Subscription id: ' + ws.subscription_id, \n",
        "      'Resource group: ' + ws.resource_group, sep = '\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Set experiment name and start the run"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "experiment_name = 'export-to-tensorboard'\n",
        "exp = Experiment(ws, experiment_name)\n",
        "root_run = exp.start_logging()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# load diabetes dataset, a well-known built-in small dataset that comes with scikit-learn\n",
        "from sklearn.datasets import load_diabetes\n",
        "from sklearn.linear_model import Ridge\n",
        "from sklearn.metrics import mean_squared_error\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X, y = load_diabetes(return_X_y=True)\n",
        "\n",
        "columns = ['age', 'gender', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']\n",
        "\n",
        "x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
        "data = {\n",
        "    \"train\":{\"x\":x_train, \"y\":y_train},        \n",
        "    \"test\":{\"x\":x_test, \"y\":y_test}\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Example experiment\n",
        "from tqdm import tqdm\n",
        "\n",
        "alphas = [.1, .2, .3, .4, .5, .6 , .7]\n",
        "\n",
        "# try a bunch of alpha values in a Linear Regression (Ridge) model\n",
        "for alpha in tqdm(alphas):\n",
        "    # create a bunch of child runs\n",
        "    with root_run.child_run(\"alpha\" + str(alpha)) as run:\n",
        "        # More data science stuff\n",
        "        reg = Ridge(alpha=alpha)\n",
        "        reg.fit(data[\"train\"][\"x\"], data[\"train\"][\"y\"])\n",
        "        # TODO save model\n",
        "        preds = reg.predict(data[\"test\"][\"x\"])\n",
        "        mse = mean_squared_error(preds, data[\"test\"][\"y\"])\n",
        "        # End train and eval\n",
        "\n",
        "        # log alpha, mean_squared_error and feature names in run history\n",
        "        root_run.log(\"alpha\", alpha)\n",
        "        root_run.log(\"mse\", mse)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Export Run History to Tensorboard logs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Export Run History to Tensorboard logs\n",
        "from azureml.contrib.tensorboard.export import export_to_tensorboard\n",
        "import os\n",
        "import tensorflow as tf\n",
        "\n",
        "logdir = 'exportedTBlogs'\n",
        "log_path = os.path.join(os.getcwd(), logdir)\n",
        "try:\n",
        "    os.stat(log_path)\n",
        "except os.error:\n",
        "    os.mkdir(log_path)\n",
        "print(logdir)\n",
        "\n",
        "# export run history for the project\n",
        "export_to_tensorboard(root_run, logdir)\n",
        "\n",
        "# or export a particular run\n",
        "# export_to_tensorboard(run, logdir)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "root_run.complete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Start Tensorboard\n",
        "\n",
        "Or you can start the Tensorboard outside this notebook to view the result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azureml.contrib.tensorboard import Tensorboard\n",
        "\n",
        "# The Tensorboard constructor takes an array of runs, so be sure and pass it in as a single-element array here\n",
        "tb = Tensorboard([], local_root=logdir, port=6006)\n",
        "\n",
        "# If successful, start() returns a string with the URI of the instance.\n",
        "tb.start()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Stop Tensorboard\n",
        "\n",
        "When you're done, make sure to call the `stop()` method of the Tensorboard object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "tb.stop()"
      ]
    }
  ],
  "metadata": {
    "authors": [
      {
        "name": "roastala"
      }
    ],
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python36"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}