diff --git a/01.getting-started/06.logging-api/06.logging-api.ipynb b/01.getting-started/06.logging-api/06.logging-api.ipynb new file mode 100644 index 00000000..52b2f328 --- /dev/null +++ b/01.getting-started/06.logging-api/06.logging-api.ipynb @@ -0,0 +1,328 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Copyright (c) Microsoft Corporation. All rights reserved.\n", + "\n", + "Licensed under the MIT License." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 06. Logging APIs\n", + "This notebook showcase various ways to use the Azure Machine Learning service run logging APIs, and view the results in the Azure portal." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "Make sure you go through the [00. Installation and Configuration](../../00.configuration.ipynb) Notebook first if you haven't. Also make sure you have tqdm and matplotlib installed in the current kernel.\n", + "\n", + "```\n", + "(myenv) $ conda install -y tqdm matplotlib\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Validate Azure ML SDK installation and get version number for debugging purposes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "install" + ] + }, + "outputs": [], + "source": [ + "from azureml.core import Experiment, Run, Workspace\n", + "import azureml.core\n", + "import numpy as np\n", + "\n", + "# Check core SDK version number\n", + "print(\"SDK version:\", azureml.core.VERSION)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initialize Workspace\n", + "\n", + "Initialize a workspace object from persisted configuration." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [ + "create workspace" + ] + }, + "outputs": [], + "source": [ + "ws = Workspace.from_config()\n", + "print('Workspace name: ' + ws.name, \n", + " 'Azure region: ' + ws.location, \n", + " 'Subscription id: ' + ws.subscription_id, \n", + " 'Resource group: ' + ws.resource_group, sep='\\n')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Set experiment\n", + "Create a new experiment (or get the one with such name)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "exp = Experiment(workspace=ws, name='logging-api-test')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Log metrics\n", + "We will start a run, and use the various logging APIs to record different types of metrics during the run." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tqdm import tqdm\n", + "\n", + "# start logging for the run\n", + "run = exp.start_logging()\n", + "\n", + "# log a string value\n", + "run.log(name='Name', value='Logging API run')\n", + "\n", + "# log a numerical value\n", + "run.log(name='Magic Number', value=42)\n", + "\n", + "# Log a list of values. Note this will generate a single-variable line chart.\n", + "run.log_list(name='Fibonacci', value=[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89])\n", + "\n", + "# create a dictionary to hold a table of values\n", + "sines = {}\n", + "sines['angle'] = []\n", + "sines['sine'] = []\n", + "\n", + "for i in tqdm(range(-10, 10)):\n", + " # log a metric value repeatedly, this will generate a single-variable line chart.\n", + " run.log(name='Sigmoid', value=1 / (1 + np.exp(-i)))\n", + " angle = i / 2.0\n", + " \n", + " # log a 2 (or more) values as a metric repeatedly. This will generate a 2-variable line chart if you have 2 numerical columns.\n", + " run.log_row(name='Cosine Wave', angle=angle, cos=np.cos(angle))\n", + " \n", + " sines['angle'].append(angle)\n", + " sines['sine'].append(np.sin(angle))\n", + "\n", + "# log a dictionary as a table, this will generate a 2-variable chart if you have 2 numerical columns\n", + "run.log_table(name='Sine Wave', value=sines)\n", + "\n", + "run.complete()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Even after the run is marked completed, you can still log things." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Log an image\n", + "This is how to log a _matplotlib_ pyplot object." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "import matplotlib.pyplot as plt\n", + "angle = np.linspace(-3, 3, 50)\n", + "plt.plot(angle, np.tanh(angle), label='tanh')\n", + "plt.legend(fontsize=12)\n", + "plt.title('Hyperbolic Tangent', fontsize=16)\n", + "plt.grid(True)\n", + "\n", + "run.log_image(name='Hyperbolic Tangent', plot=plt)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Upload a file" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also upload an abitrary file. First, let's create a dummy file locally." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%%writefile myfile.txt\n", + "\n", + "This is a dummy file." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's upload this file into the run record as a run artifact, and display the properties after the upload." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "props = run.upload_file(name='myfile_in_the_cloud.txt', path_or_stream='./myfile.txt')\n", + "props.serialize()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Examine the run" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now let's take a look at the run detail page in Azure portal. Make sure you checkout the various charts and plots generated/uploaded." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "run" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can get all the metrics in that run back." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "run.get_metrics()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also see the files uploaded for this run." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "run.get_file_names()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also download all the files locally." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "os.makedirs('files', exist_ok=True)\n", + "\n", + "for f in run.get_file_names():\n", + " dest = os.path.join('files', f.split('/')[-1])\n", + " print('Downloading file {} to {}...'.format(f, dest))\n", + " run.download_file(f, dest) " + ] + } + ], + "metadata": { + "authors": [ + { + "name": "haining" + } + ], + "kernelspec": { + "display_name": "Python 3.6", + "language": "python", + "name": "python36" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.6" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} \ No newline at end of file