fix links

Merge pull request #29 from rastala/master
onnx update
2025-12-20 17:45:10 -05:00 · 2018-09-28 17:24:37 -05:00 · 2018-09-28 16:42:09 -04:00 · 2018-09-28 16:41:21 -04:00 · 2018-09-27 12:52:47 -04:00 · 2018-09-27 12:50:38 -04:00
16 changed files with 895 additions and 342 deletions
--- a/00.configuration.ipynb
+++ b/00.configuration.ipynb
@@ -14,32 +14,39 @@
   "metadata": {},
   "source": [
    "# 00. Installation and configuration\n",
    "This notebook configures your library of notebooks to connect to an Azure Machine Learning Workspace.  In this case, a library contains all of the notebooks in the current folder and any nested folders.  You can configure this notebook to use an existing workspace or create a new workspace.\n",
    "\n",
-    "## Prerequisites:\n",
+    "## What is an Azure ML Workspace and why do I need one?\n",
    "\n",
-    "### 1. Install Azure ML SDK\n",
+    "An AML Workspace is an Azure resource that organizes and coordinates the actions of many other Azure resources to assist in executing and sharing machine learning workflows.  In particular, an AML Workspace coordinates storage, databases, and compute resources providing added functionality for machine learning experimentation, operationalization, and the monitoring of operationalized models."
-    "Follow [SDK installation instructions](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-environment).\n",
+   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Access Azure Subscription\n",
    "\n",
-    "### 2. Install some additional packages\n",
+    "In order to create an AML Workspace, first you need access to an Azure Subscription.  You can [create your own](https://azure.microsoft.com/en-us/free/) or get your existing subscription information from the [Azure portal](https://portal.azure.com).\n",
-    "This Notebook requires some additional libraries. In the conda environment, run below commands: \n",
+    "\n",
-    "```shell\n",
+    "### 2. If you're running on your own local environment, install Azure ML SDK and other libraries\n",
    "\n",
    "If you are running in your own environment, follow [SDK installation instructions](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-environment).  If you are running in Azure Notebooks or another Microsoft managed environment, the SDK is already installed.\n",
    "\n",
    "Also install following libraries to your environment. Many of the example notebooks depend on them\n",
    "\n",
    "```\n",
    "(myenv) $ conda install -y matplotlib tqdm scikit-learn\n",
    "```\n",
    "\n",
-    "### 3. Make sure your subscription is registered to use ACI.\n",
+    "Once installation is complete, check the Azure ML SDK version:"
    "This Notebook makes use of Azure Container Instance (ACI). You need to ensure your subscription has been registered to use ACI in order be able to deploy a dev/test web service.\n",
    "```shell\n",
    "# check to see if ACI is already registered\n",
    "(myenv) $ az provider show -n Microsoft.ContainerInstance -o table\n",
    "\n",
    "# if ACI is not registered, run this command.\n",
    "# note you need to be the subscription owner in order to execute this command successfully.\n",
    "(myenv) $ az provider register -n Microsoft.ContainerInstance\n",
    "```\n",
    "\n",
    "In this example you will optionally create an Azure Machine Learning Workspace and initialize your notebook directory to easily use this workspace.  Typically you will only need to run this once per notebook directory, and all other notebooks in this directory or any sub-directories will automatically use the settings you indicate here.\n",
    "\n",
    "This notebook also contains optional cells to install and update the require Azure Machine Learning libraries."
   ]
  },
  {
@@ -52,7 +59,6 @@
   },
   "outputs": [],
   "source": [
    "# Check core SDK version number for debugging purposes\n",
    "import azureml.core\n",
    "\n",
    "print(\"SDK Version:\", azureml.core.VERSION)"
@@ -62,20 +68,63 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Initialize an Azure ML Workspace\n",
+    "### 3. Make sure your subscription is registered to use ACI\n",
-    "### What is an Azure ML Workspace and why do I need one?\n",
+    "Azure Machine Learning makes use of Azure Container Instance (ACI). You need to ensure your subscription has been registered to use ACI in order be able to deploy a dev/test web service.  If you have run through the quickstart experience you have already performed this step.  Otherwise you will need to use the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) and execute the following commands.\n",
    "\n",
-    "An AML Workspace is an Azure resource that organaizes and coordinates the actions of many other Azure resources to assist in executing and sharing machine learning workflows.  In particular, an AML Workspace coordinates storage, databases, and compute resources providing added functionality for machine learning experimentation, operationalization, and the monitoring of operationalized models.\n",
+    "```shell\n",
    "# check to see if ACI is already registered\n",
    "(myenv) $ az provider show -n Microsoft.ContainerInstance -o table\n",
    "\n",
-    "### What do I need\n",
+    "# if ACI is not registered, run this command.\n",
    "# note you need to be the subscription owner in order to execute this command successfully.\n",
    "(myenv) $ az provider register -n Microsoft.ContainerInstance\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up your Azure Machine Learning workspace\n",
    "\n",
-    "In order to use an AML Workspace, first you need access to an Azure Subscription.  You can [create your own](https://azure.microsoft.com/en-us/free/) or get your existing subscription information from the [Azure portal](https://portal.azure.com).  Inside your subscription, you will need access to a _resource group_, which organizes Azure resources and provides a default region for the resources in a group.  You can see what resource groups to which you have access, or create a new one in the [Azure portal](https://portal.azure.com)\n",
+    "### Option 1: You have workspace already\n",
    "If you ran the Azure Machine Learning [quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-get-started) in Azure Notebooks, you already have a configured workspace!  You can go to your Azure Machine Learning Getting Started library, view *config.json* file, and copy-paste the values for subscription ID, resource group and workspace name below.\n",
    "\n",
-    "You can also easily create a new resource group using azure-cli.\n",
+    "If you have a workspace created another way, [these instructions](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-environment#create-workspace-configuration-file) describe how to get your subscription and workspace information.\n",
    "\n",
-    "```sh\n",
+    "If this cell succeeds, you're done configuring this library!  Otherwise continue to follow the instructions in the rest of the notebook."
-    "(myenv) $ az group create -n my_resource_group -l eastus2\n",
+   ]
-    "```\n",
+  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Workspace\n",
    "\n",
    "subscription_id ='<subscription-id>'\n",
    "resource_group ='<resource-group>'\n",
    "workspace_name = '<workspace-name>'\n",
    "\n",
    "try:\n",
    "   ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name = workspace_name)\n",
    "   ws.write_config()\n",
    "   print('Workspace configuration succeeded. You are all set!')\n",
    "except:\n",
    "   print('Workspace not found. Run the cells below.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 2: You don't have workspace yet\n",
    "\n",
    "\n",
    "#### Requirements\n",
    "\n",
    "Inside your Azure subscription, you will need access to a _resource group_, which organizes Azure resources and provides a default region for the resources in a group.  You can see what resource groups to which you have access, or create a new one in the [Azure portal](https://portal.azure.com).  If you don't have a resource group, the create workspace command will create one for you using the name you provide.\n",
    "\n",
    "To create or access an Azure ML Workspace, you will need to import the AML library and the following information:\n",
    "* A name for your workspace\n",
@@ -89,8 +138,17 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "### Supported Azure Regions\n",
+    "#### Supported Azure Regions\n",
-    "Please specify the Azure subscription Id, resource group name, workspace name, and the region in which you want to create the workspace, for example \"eastus2\". "
+    "Specify a region where your workspace will be located from the list of [Azure Machine Learning regions](https://linktoregions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "workspace_region = \"eastus2\""
   ]
  },
  {
@@ -101,21 +159,22 @@
   "source": [
    "import os\n",
    "\n",
-    "subscription_id = os.environ.get(\"SUBSCRIPTION_ID\", \"<my-subscription-id>\")\n",
+    "subscription_id = os.environ.get(\"SUBSCRIPTION_ID\", subscription_id)\n",
-    "resource_group = os.environ.get(\"RESOURCE_GROUP\", \"<my-rg>\")\n",
+    "resource_group = os.environ.get(\"RESOURCE_GROUP\", resource_group)\n",
-    "workspace_name = os.environ.get(\"WORKSPACE_NAME\", \"<my-workspace>\")\n",
+    "workspace_name = os.environ.get(\"WORKSPACE_NAME\", workspace_name)\n",
-    "workspace_region = os.environ.get(\"WORKSPACE_REGION\", \"eastus2\")"
+    "workspace_region = os.environ.get(\"WORKSPACE_REGION\", workspace_region)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Creating a workspace\n",
+    "#### Create the workspace\n",
-    "If you already have access to an AML Workspace you want to use, you can skip this cell.  Otherwise, this cell will create an AML workspace for you in a subscription provided you have the correct permissions.\n",
+    "This cell will create an AML workspace for you in a subscription provided you have the correct permissions.\n",
    "\n",
    "This will fail when:\n",
    "1. You do not have permission to create a workspace in the resource group\n",
    "2. You do not have permission to create a resource group if it's non-existing.\n",
    "2. You are not a subscription owner or contributor and no Azure ML workspaces have ever been created in this subscription\n",
    "\n",
    "If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to provision the required resources."
@@ -138,58 +197,12 @@
    "                      subscription_id = subscription_id,\n",
    "                      resource_group = resource_group, \n",
    "                      location = workspace_region,\n",
    "                      create_resource_group = True,\n",
    "                      exist_ok = True)\n",
-    "ws.get_details()"
+    "ws.get_details()\n",
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configuring your local environment\n",
    "You can validate that you have access to the specified workspace and write a configuration file to the default configuration location, `./aml_config/config.json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "create workspace"
    ]
   },
   "outputs": [],
   "source": [
    "ws = Workspace(workspace_name = workspace_name,\n",
    "               subscription_id = subscription_id,\n",
    "               resource_group = resource_group)\n",
    "\n",
    "# persist the subscription id, resource group name, and workspace name in aml_config/config.json.\n",
    "ws.write_config()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can then load the workspace from this config file from any notebook in the current directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "create workspace"
    ]
   },
   "outputs": [],
   "source": [
    "# load workspace configuratio from ./aml_config/config.json file.\n",
    "my_workspace = Workspace.from_config()\n",
    "my_workspace.get_details()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
@@ -215,7 +228,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.6.4"
+   "version": "3.6.6"
  }
 },
 "nbformat": 4,
--- a/01.getting-started/01.train-within-notebook/01.train-within-notebook.ipynb
+++ b/01.getting-started/01.train-within-notebook/01.train-within-notebook.ipynb
@@ -457,7 +457,8 @@
   },
   "outputs": [],
   "source": [
-    "models = ws.models(name='best_model')\n",
+    "from azureml.core.model import Model\n",
    "models = Model.list(workspace=ws, name='best_model')\n",
    "for m in models:\n",
    "    print(m.name, m.version)"
   ]
@@ -526,6 +527,7 @@
    "\n",
    "myenv = CondaDependencies()\n",
    "myenv.add_conda_package(\"scikit-learn\")\n",
    "myenv.add_pip_package(\"pynacl==1.2.1\")\n",
    "print(myenv.serialize_to_string())\n",
    "\n",
    "with open(\"myenv.yml\",\"w\") as f:\n",
--- a/01.getting-started/04.train-on-remote-vm/04.train-on-remote-vm.ipynb
+++ b/01.getting-started/04.train-on-remote-vm/04.train-on-remote-vm.ipynb
@@ -13,12 +13,16 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "# 04. Train in a remote VM (MLC managed DSVM)\n",
+    "# 04. Train in a remote Linux VM\n",
    "* Create Workspace\n",
    "* Create Project\n",
    "* Create `train.py` file\n",
-    "* Create DSVM as Machine Learning Compute (MLC) resource\n",
+    "* Create (or attach) DSVM as compute resource.\n",
-    "* Configure & execute a run in a conda environment in the default miniconda Docker container on DSVM"
+    "* Upoad data files into default datastore\n",
    "* Configure & execute a run in a few different ways\n",
    "    - Use system-built conda\n",
    "    - Use existing Python environment\n",
    "    - Use Docker \n",
    "* Find the best model in the run"
   ]
  },
  {
@@ -59,7 +63,7 @@
    "from azureml.core import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
-    "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
+    "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\\n')"
   ]
  },
  {
@@ -80,17 +84,14 @@
    "experiment_name = 'train-on-remote-vm'\n",
    "\n",
    "from azureml.core import Experiment\n",
-    "\n",
+    "exp = Experiment(workspace=ws, name=experiment_name)"
    "exp = Experiment(workspace = ws, name = experiment_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## View `train.py`\n",
+    "Let's also create a local folder to hold the training script."
    "\n",
    "For convenience, we created a training script for you. It is printed below as a text, but you can also run `%pfile ./train.py` in a cell to show the file."
   ]
  },
  {
@@ -99,7 +100,87 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "with open('./train.py', 'r') as training_script:\n",
+    "import os\n",
    "script_folder = './vm-run'\n",
    "os.makedirs(script_folder, exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload data files into datastore\n",
    "Every workspace comes with a default datastore (and you can register more) which is backed by the Azure blob storage account associated with the workspace. We can use it to transfer data from local to the cloud, and access it from the compute target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default datastore\n",
    "ds = ws.get_default_datastore()\n",
    "print(ds.name, ds.datastore_type, ds.account_name, ds.container_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load diabetes data from `scikit-learn` and save it as 2 local files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_diabetes\n",
    "import numpy as np\n",
    "\n",
    "training_data = load_diabetes()\n",
    "np.save(file='./feeatures.npy', arr=training_data['data'])\n",
    "np.save(file='./labels.npy', arr=training_data['target'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's upload the 2 files into the default datastore under a path named `diabetes`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds.upload_files(['./feeatures.npy', './labels.npy'], target_path='diabetes', overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View `train.py`\n",
    "\n",
    "For convenience, we created a training script for you. It is printed below as a text, but you can also run `%pfile ./train.py` in a cell to show the file. Please pay special attention on how we are loading the features and labels from files in the `data_folder` path, which is passed in as an argument of the training script (shown later)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# copy train.py into the script folder\n",
    "import shutil\n",
    "shutil.copy('./train.py', os.path.join(script_folder, 'train.py'))\n",
    "\n",
    "with open(os.path.join(script_folder, './train.py'), 'r') as training_script:\n",
    "    print(training_script.read())"
   ]
  },
@@ -111,7 +192,7 @@
    "\n",
    "**Note**: If creation fails with a message about Marketplace purchase eligibilty, go to portal.azure.com, start creating DSVM there, and select \"Want to create programmatically\" to enable programmatic creation. Once you've enabled it, you can exit without actually creating VM.\n",
    " \n",
-    "**Note**: By default SSH runs on port 22 and you don't need to specify it. But if for security reasons you switch to a different port (such as 5022), you can append the port number to the address like the example below."
+    "**Note**: By default SSH runs on port 22 and you don't need to specify it. But if for security reasons you switch to a different port (such as 5022), you can specify the port number in the provisioning configuration object."
   ]
  },
  {
@@ -126,20 +207,21 @@
    "compute_target_name = 'mydsvm'\n",
    "\n",
    "try:\n",
-    "    dsvm_compute = DsvmCompute(workspace = ws, name = compute_target_name)\n",
+    "    dsvm_compute = DsvmCompute(workspace=ws, name=compute_target_name)\n",
    "    print('found existing:', dsvm_compute.name)\n",
    "except ComputeTargetException:\n",
    "    print('creating new.')\n",
-    "    dsvm_config = DsvmCompute.provisioning_configuration(vm_size = \"Standard_D2_v2\")\n",
+    "    dsvm_config = DsvmCompute.provisioning_configuration(vm_size=\"Standard_D2_v2\")\n",
-    "    dsvm_compute = DsvmCompute.create(ws, name = compute_target_name, provisioning_configuration = dsvm_config)\n",
+    "    dsvm_compute = DsvmCompute.create(ws, name=compute_target_name, provisioning_configuration=dsvm_config)\n",
-    "    dsvm_compute.wait_for_completion(show_output = True)"
+    "    dsvm_compute.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Attach an existing Linux DSVM as a compute target\n"
+    "## Attach an existing Linux DSVM\n",
    "You can also attach an existing Linux VM as a compute target. The default port is 22."
   ]
  },
  {
@@ -148,18 +230,220 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "'''\n",
+    "from azureml.core.compute import RemoteCompute \n",
-    " from azureml.core.compute import RemoteCompute \n",
+    "# if you want to connect using SSH key instead of username/password you can provide parameters private_key_file and private_key_passphrase \n",
-    " # if you want to connect using SSH key instead of username/password you can provide parameters private_key_file and private_key_passphrase \n",
+    "attached_dsvm_compute = RemoteCompute.attach(workspace=ws,\n",
-    " dsvm_compute = RemoteCompute.attach(ws,name=\"attach-from-sdk6\",username=<username>,address=<ipaddress>,ssh_port=22,password=<password>)\n",
+    "                                             name=\"attached_vm\",\n",
-    "'''"
+    "                                             username='<usename>',\n",
    "                                             address='<ip_adress_or_fqdn>',\n",
    "                                             ssh_port=22,\n",
    "                                             password='<password>')\n",
    "attached_dsvm_compute.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Configure & Run"
+    "## Configure & Run\n",
    "First let's create a `DataReferenceConfigruation` object to inform the system what data folder to download to the copmute target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.runconfig import DataReferenceConfiguration\n",
    "dr = DataReferenceConfiguration(datastore_name=ds.name, \n",
    "                   path_on_datastore='diabetes', \n",
    "                   mode='download', # download files from datastore to compute target\n",
    "                   overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can try a few different ways to run the training script in the VM."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conda run\n",
    "You can ask the system to build a conda environment based on your dependency specification, and submit your script to run there. Once the environment is built, and if you don't change your dependencies, it will be reused in subsequent runs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.runconfig import RunConfiguration\n",
    "from azureml.core.conda_dependencies import CondaDependencies\n",
    "\n",
    "# create a new RunConfig object\n",
    "conda_run_config = RunConfiguration(framework=\"python\")\n",
    "\n",
    "# Set compute target to the Linux DSVM\n",
    "conda_run_config.target = dsvm_compute.name\n",
    "\n",
    "# set the data reference of the run coonfiguration\n",
    "conda_run_config.data_references = {ds.name: dr}\n",
    "\n",
    "# specify CondaDependencies obj\n",
    "conda_run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Run\n",
    "from azureml.core import ScriptRunConfig\n",
    "\n",
    "src = ScriptRunConfig(source_directory=script_folder, \n",
    "                      script='train.py', \n",
    "                      run_config=conda_run_config, \n",
    "                      # pass the datastore reference as a parameter to the training script\n",
    "                      arguments=['--data-folder', str(ds.as_download())] \n",
    "                     ) \n",
    "run = exp.submit(config=src)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show the run object. You can navigate to the Azure portal to see detailed information about the run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Native VM run\n",
    "You can also configure to use an exiting Python environment in the VM to execute the script without asking the system to create a conda environment for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new RunConfig object\n",
    "vm_run_config = RunConfiguration(framework=\"python\")\n",
    "\n",
    "# Set compute target to the Linux DSVM\n",
    "vm_run_config.target = dsvm_compute.name\n",
    "\n",
    "# set the data reference of the run coonfiguration\n",
    "conda_run_config.data_references = {ds.name: dr}\n",
    "\n",
    "# Let system know that you will configure the Python environment yourself.\n",
    "vm_run_config.environment.python.user_managed_dependencies = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The below run will likely fail because `train.py` needs dependency `azureml`, `scikit-learn` and others, which are not found in that Python environment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "src = ScriptRunConfig(source_directory=script_folder, \n",
    "                      script='train.py', \n",
    "                      run_config=vm_run_config,\n",
    "                      # pass the datastore reference as a parameter to the training script\n",
    "                      arguments=['--data-folder', str(ds.as_download())])\n",
    "run = exp.submit(config=src)\n",
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "You can choose to SSH into the VM and install Azure ML SDK, and any other missing dependencies, in that Python environment. For demonstration purposes, we simply are going to create another script `train2.py` that doesn't have azureml dependencies, and submit it instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $script_folder/train2.py\n",
    "print('####################################')\n",
    "print('Hello World (without Azure ML SDK)!')\n",
    "print('####################################')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try again. And this time it should work fine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "src = ScriptRunConfig(source_directory=script_folder, \n",
    "                      script='train2.py', \n",
    "                      run_config=vm_run_config)\n",
    "run = exp.submit(config=src)\n",
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note even in this case you get a run record with some basic statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run"
   ]
  },
  {
@@ -167,7 +451,7 @@
   "metadata": {},
   "source": [
    "### Configure a Docker run with new conda environment on the VM\n",
-    "You can execute in a Docker container in the VM. If you choose this route, you don't need to install anything on the VM yourself. Azure ML execution service will take care of it for you."
+    "You can execute in a Docker container in the VM. If you choose this option, the system will pull down a base Docker image, build a new conda environment in it if you ask for (you can also skip this if you are using a customer Docker image when a preconfigured Python environment), start a container, and run your script in there. This image is also uploaded into your ACR (Azure Container Registry) assoicated with your workspace, an reused if your dependencies don't change in the subsequent runs."
   ]
  },
  {
@@ -181,26 +465,23 @@
    "\n",
    "\n",
    "# Load the \"cpu-dsvm.runconfig\" file (created by the above attach operation) in memory\n",
-    "run_config = RunConfiguration(framework = \"python\")\n",
+    "docker_run_config = RunConfiguration(framework=\"python\")\n",
    "\n",
    "# Set compute target to the Linux DSVM\n",
-    "run_config.target = compute_target_name\n",
+    "docker_run_config.target = dsvm_compute.name\n",
    "\n",
    "# Use Docker in the remote VM\n",
-    "run_config.environment.docker.enabled = True\n",
+    "docker_run_config.environment.docker.enabled = True\n",
    "\n",
    "# Use CPU base image from DockerHub\n",
-    "run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE\n",
+    "docker_run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE\n",
-    "print('Base Docker image is:', run_config.environment.docker.base_image)\n",
+    "print('Base Docker image is:', docker_run_config.environment.docker.base_image)\n",
    "\n",
-    "# Ask system to provision a new one based on the conda_dependencies.yml file\n",
+    "# set the data reference of the run coonfiguration\n",
-    "run_config.environment.python.user_managed_dependencies = False\n",
+    "docker_run_config.data_references = {ds.name: dr}\n",
    "\n",
    "# Prepare the Docker and conda environment automatically when executingfor the first time.\n",
    "run_config.prepare_environment = True\n",
    "\n",
    "# specify CondaDependencies obj\n",
-    "run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
+    "docker_run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
   ]
  },
  {
@@ -217,11 +498,21 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "from azureml.core import Run\n",
+    "src = ScriptRunConfig(source_directory=script_folder, \n",
-    "from azureml.core import ScriptRunConfig\n",
+    "                      script='train.py', \n",
-    "\n",
+    "                      run_config=docker_run_config,\n",
-    "src = ScriptRunConfig(source_directory = '.', script = 'train.py', run_config = run_config)\n",
+    "                      # pass the datastore reference as a parameter to the training script\n",
-    "run = exp.submit(src)"
+    "                      arguments=['--data-folder', str(ds.as_download())])\n",
    "run = exp.submit(config=src)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
@@ -241,19 +532,17 @@
   ]
  },
  {
-   "cell_type": "code",
+   "cell_type": "markdown",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
-    "run.wait_for_completion(show_output = True)"
+    "### Find the best model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "### Find the best run"
+    "Now we have tried various execution modes, we can find the best model from the last run."
   ]
  },
  {
@@ -273,10 +562,13 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "import numpy as np\n",
+    "# find the index where MSE is the smallest\n",
    "indices = list(range(0, len(metrics['mse'])))\n",
    "min_mse_index = min(indices, key=lambda x: metrics['mse'][x])\n",
    "\n",
    "print('When alpha is {1:0.2f}, we have min MSE {0:0.2f}.'.format(\n",
-    "    min(metrics['mse']), \n",
+    "    metrics['mse'][min_mse_index], \n",
-    "    metrics['alpha'][np.argmin(metrics['mse'])]\n",
+    "    metrics['alpha'][min_mse_index]\n",
    "))"
   ]
  },
@@ -313,7 +605,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.6"
  }
 },
 "nbformat": 4,
--- a/01.getting-started/04.train-on-remote-vm/train.py
+++ b/01.getting-started/04.train-on-remote-vm/train.py
@@ -2,7 +2,8 @@
 # Licensed under the MIT license.
 import os
-from sklearn.datasets import load_diabetes
+import argparse
 from sklearn.linear_model import Ridge
 from sklearn.metrics import mean_squared_error
 from sklearn.model_selection import train_test_split
@@ -12,8 +13,16 @@ from sklearn.externals import joblib
 import numpy as np
 os.makedirs('./outputs', exist_ok=True)
 parser = argparse.ArgumentParser()
 parser.add_argument('--data-folder', type=str,
                    dest='data_folder', help='data folder')
 args = parser.parse_args()
-X, y = load_diabetes(return_X_y=True)
+print('Data folder is at:', args.data_folder)
 print('List all files: ', os.listdir(args.data_folder))
 X = np.load(os.path.join(args.data_folder, 'features.npy'))
 y = np.load(os.path.join(args.data_folder, 'labels.npy'))
 run = Run.get_submitted_run()
--- a/01.getting-started/05.train-in-spark/05.train-in-spark.ipynb
+++ b/01.getting-started/05.train-in-spark/05.train-in-spark.ipynb
@@ -58,7 +58,7 @@
    "from azureml.core import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
-    "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
+    "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\\n')"
   ]
  },
  {
@@ -77,8 +77,7 @@
    "experiment_name = 'train-on-spark'\n",
    "\n",
    "from azureml.core import Experiment\n",
-    "\n",
+    "exp = Experiment(workspace=ws, name=experiment_name)"
    "exp = Experiment(workspace = ws, name = experiment_name)"
   ]
  },
  {
@@ -107,12 +106,86 @@
    "## Configure & Run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure an ACI run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.runconfig import RunConfiguration\n",
    "from azureml.core.conda_dependencies import CondaDependencies\n",
    "\n",
    "# use pyspark framework\n",
    "aci_run_config = RunConfiguration(framework=\"pyspark\")\n",
    "\n",
    "# use ACI to run the Spark job\n",
    "aci_run_config.target = 'containerinstance'\n",
    "aci_run_config.container_instance.region = 'eastus2'\n",
    "aci_run_config.container_instance.cpu_cores = 1\n",
    "aci_run_config.container_instance.memory_gb = 2\n",
    "\n",
    "# specify base Docker image to use\n",
    "aci_run_config.environment.docker.enabled = True\n",
    "aci_run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_MMLSPARK_CPU_IMAGE\n",
    "\n",
    "# specify CondaDependencies\n",
    "cd = CondaDependencies()\n",
    "cd.add_conda_package('numpy')\n",
    "aci_run_config.environment.python.conda_dependencies = cd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Submit script to ACI to run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import ScriptRunConfig\n",
    "\n",
    "script_run_config = ScriptRunConfig(source_directory = '.',\n",
    "                                    script= 'train-spark.py',\n",
    "                                    run_config = aci_run_config)\n",
    "run = exp.submit(script_run_config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Attach an HDI cluster\n",
-    "To use HDI commpute target:\n",
+    "Now we can use a real Spark cluster, HDInsight for Spark, to run this job. To use HDI commpute target:\n",
    " 1. Create an Spark for HDI cluster in Azure. Here is some [quick instructions](https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/how-to-create-dsvm-hdi). Make sure you use the Ubuntu flavor, NOT CentOS.\n",
    " 2. Enter the IP address, username and password below"
   ]
@@ -124,22 +197,22 @@
   "outputs": [],
   "source": [
    "from azureml.core.compute import HDInsightCompute\n",
    "from azureml.exceptions import ComputeTargetException\n",
    "\n",
    "try:\n",
    "    # if you want to connect using SSH key instead of username/password you can provide parameters private_key_file and private_key_passphrase\n",
-    "    hdi_compute_new = HDInsightCompute.attach(ws, \n",
+    "    hdi_compute = HDInsightCompute.attach(workspace=ws, \n",
-    "                             name=\"hdi-attach\", \n",
+    "                                          name=\"myhdi\", \n",
-    "                             address=\"hdi-ignite-demo-ssh.azurehdinsight.net\", \n",
+    "                                          address=\"myhdi-ssh.azurehdinsight.net\", \n",
-    "                             ssh_port=22, \n",
+    "                                          ssh_port=22, \n",
-    "                             username='<username>', \n",
+    "                                          username='<ssh-username>', \n",
-    "                             password='<password>')\n",
+    "                                          password='<ssh-pwd>')\n",
    "\n",
-    "except UserErrorException as e:\n",
+    "except ComputeTargetException as e:\n",
    "    print(\"Caught = {}\".format(e.message))\n",
    "    print(\"Compute config already attached.\")\n",
    "    \n",
-    "    \n",
+    "        \n",
-    "hdi_compute_new.wait_for_completion(show_output=True)"
+    "hdi_compute.wait_for_completion(show_output=True)"
   ]
  },
  {
@@ -160,27 +233,18 @@
    "\n",
    "\n",
    "# Load the \"cpu-dsvm.runconfig\" file (created by the above attach operation) in memory\n",
-    "run_config = RunConfiguration(framework = \"python\")\n",
+    "hdi_run_config = RunConfiguration(framework=\"pyspark\")\n",
    "\n",
    "# Set compute target to the Linux DSVM\n",
-    "run_config.target = hdi_compute.name\n",
+    "hdi_run_config.target = hdi_compute.name\n",
    "\n",
    "# Use Docker in the remote VM\n",
    "# run_config.environment.docker.enabled = True\n",
    "\n",
    "# Use CPU base image from DockerHub\n",
    "# run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE\n",
    "# print('Base Docker image is:', run_config.environment.docker.base_image)\n",
    "\n",
    "# Ask system to provision a new one based on the conda_dependencies.yml file\n",
-    "run_config.environment.python.user_managed_dependencies = False\n",
+    "hdi_run_config.environment.python.user_managed_dependencies = False\n",
    "\n",
    "# Prepare the Docker and conda environment automatically when executingfor the first time.\n",
    "# run_config.prepare_environment = True\n",
    "\n",
    "# specify CondaDependencies obj\n",
-    "# run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])\n",
+    "cd = CondaDependencies()\n",
-    "# load the runconfig object from the \"myhdi.runconfig\" file generated by the attach operaton above."
+    "cd.add_conda_package('numpy')\n",
    "hdi_run_config.environment.python.conda_dependencies = cd"
   ]
  },
  {
@@ -196,10 +260,12 @@
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import ScriptRunConfig\n",
    "\n",
    "script_run_config = ScriptRunConfig(source_directory = '.',\n",
    "                                    script= 'train-spark.py',\n",
-    "                                    run_config = run_config)\n",
+    "                                    run_config = hdi_run_config)\n",
-    "run = experiment.submit(script_run_config)"
+    "run = exp.submit(script_run_config)"
   ]
  },
  {
@@ -218,7 +284,9 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "run.wait_for_completion(show_output = True)"
+    "# get all metris logged in the run\n",
    "metrics = run.get_metrics()\n",
    "print(metrics)"
   ]
  },
  {
@@ -226,18 +294,14 @@
   "execution_count": null,
   "metadata": {},
   "outputs": [],
-   "source": [
+   "source": []
    "# get all metris logged in the run\n",
    "metrics = run.get_metrics()\n",
    "print(metrics)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
-   "display_name": "Python 3.6",
+   "display_name": "Python 3",
   "language": "python",
-   "name": "python36"
+   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
@@ -249,7 +313,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.6"
  }
 },
 "nbformat": 4,
--- a/01.getting-started/10.register-model-create-image-deploy-service/10.register-model-create-image-deploy-service.ipynb
+++ b/01.getting-started/10.register-model-create-image-deploy-service/10.register-model-create-image-deploy-service.ipynb
@@ -129,9 +129,9 @@
   },
   "outputs": [],
   "source": [
-    "regression_models = ws.models(tags=['area'])\n",
+    "regression_models = Model.list(workspace=ws, tags=['area'])\n",
-    "for name, m in regression_models.items():\n",
+    "for m in regression_models:\n",
-    "    print(\"Name:\", name,\"\\tVersion:\", m.version, \"\\tDescription:\", m.description, m.tags)"
+    "    print(\"Name:\", m.name,\"\\tVersion:\", m.version, \"\\tDescription:\", m.description, m.tags)"
   ]
  },
  {
@@ -206,6 +206,7 @@
    "from azureml.core.conda_dependencies import CondaDependencies \n",
    "\n",
    "myenv = CondaDependencies.create(conda_packages=['numpy','scikit-learn'])\n",
    "myenv.add_pip_package(\"pynacl==1.2.1\")\n",
    "\n",
    "with open(\"myenv.yml\",\"w\") as f:\n",
    "    f.write(myenv.serialize_to_string())"
--- a/README.md
+++ b/README.md
@@ -5,27 +5,23 @@ To run the notebooks in this repository use one of these methods:
 ## Use Azure Notebooks - Jupyter based notebooks in the Azure cloud
 1. [![Azure Notebooks](https://notebooks.azure.com/launch.png)](https://aka.ms/aml-clone-azure-notebooks)
-[Import sample notebooks ](https://aka.ms/aml-clone-azure-notebooks) into Azure Notebooks if they are not already there.
+[Import sample notebooks ](https://aka.ms/aml-clone-azure-notebooks) into Azure Notebooks.
-1.  Create a workspace and its configuration file (**config.json**) using [these instructions](https://aka.ms/aml-how-to-configure-environment).
+1. Follow the instructions in the [00.configuration](00.configuration.ipynb) notebook to create and connect to a workspace.
-1. Select `+New` in the Azure Notebook toolbar to add your **config.json** file to the imported folder.
+1. Open one of the sample notebooks.
    ![upload config file to notebook folder](images/additems.png)
 1. Open the notebook.
-    **Make sure the Azure Notebook kernal is set to `Python 3.6`** when you open a notebook. 
+    **Make sure the Azure Notebook kernel is set to `Python 3.6`** when you open a notebook.  
-    ![set kernal to Python 3.6](images/python36.png)
+    ![set kernel to Python 3.6](images/python36.png)
 ## **Use your own notebook server**
-1. Use [these instructions](https://aka.ms/aml-how-to-configure-environment) to:
+1. Setup a Jupyter Notebook server and [install the Azure Machine Learning SDK](https://aka.ms/aml-how-to-configure-environment).
    * Create a workspace and its configuration file (**config.json**).
    * Configure your notebook server.
 1. Clone [this repository](https://aka.ms/aml-notebooks).
 1. Add your **config.json** file to the cloned folder
 1. You may need to install other packages for specific notebooks
 1. Start your notebook server.
-1. Open the notebook you want to run.
+1. Follow the instructions in the [00.configuration](00.configuration.ipynb) notebook to create and connect to a workspace.
 1. Open one of the sample notebooks.
 > Note: **Looking for automated machine learning samples?**
 > For your convenience, you can use an installation script instead of the steps below for the automated ML notebooks. Go to the [automl folder README](automl/README.md) and follow the instructions.  The script installs all  packages needed for notebooks in that folder.
--- a/automl/README.md
+++ b/automl/README.md
@@ -20,15 +20,13 @@ If you are an experienced data scientist, AutoML will help increase your product
 ## Running samples in Azure Notebooks - Jupyter based notebooks in the Azure cloud 
 1. [![Azure Notebooks](https://notebooks.azure.com/launch.png)](https://aka.ms/aml-clone-azure-notebooks)
-[Import sample notebooks ](https://aka.ms/aml-clone-azure-notebooks) into Azure Notebooks if they are not already there.
+[Import sample notebooks ](https://aka.ms/aml-clone-azure-notebooks) into Azure Notebooks.
-1.  Create a workspace and its configuration file (**config.json**) using [these instructions](https://aka.ms/aml-how-to-configure-environment).
+1. Follow the instructions in the [../00.configuration](00.configuration.ipynb) notebook to create and connect to a workspace.
-1. Select `+New` in the Azure Notebook toolbar to add your **config.json** file to the imported folder.
+1. Open one of the sample notebooks.
    ![upload config file to notebook folder](../images/additems.png)
 1. Open the notebook.
-    **Make sure the Azure Notebook kernal is set to `Python 3.6`** when you open a notebook. 
+    **Make sure the Azure Notebook kernel is set to `Python 3.6`** when you open a notebook. 
-    ![set kernal to Python 3.6](../images/python36.png)
+    ![set kernel to Python 3.6](../images/python36.png)
 <a name="localconda"></a>
 ## Running samples in a Local Conda environment 
--- a/databricks/Databricks_AMLSDK_github.dbc
+++ b/databricks/Databricks_AMLSDK_github.dbc
--- a/databricks/readme.md
+++ b/databricks/readme.md
@@ -1,9 +1,9 @@
-# Azure Databricks - Azure ML SDK Sample Notebooks
+# Azure Databricks - Azure Machine Learning SDK Sample Notebooks
-**NOTE**: With the latest version of our AML SDK, there are some API changes due to which previous version of notebooks will not work.
+**NOTE**: With the latest version of Azure Machine Learning SDK, there are some API changes due to which previous version of notebooks will not work.
-Kindly use this v4 notebooks (updated Sep 18)– if you had installed the AML SDK in your Databricks cluster please update to latest SDK version by installing azureml-sdk[databricks] as a library from GUI. 
+Please remove the previous SDK version and install the latest SDK by installing **azureml-sdk[databricks]** as a PyPi library in Azure Databricks workspace. 
-**NOTE**: Please create your Azure Databricks cluster as v4.x (high concurrency preferred) with **Python 3** (dropdown). We are extending it to more runtimes asap.
+**NOTE**: Please create your Azure Databricks cluster as v4.x (high concurrency preferred) with **Python 3** (dropdown).
 **NOTE**: Some packages like psutil upgrade libs that can cause a conflict, please install such packages by freezing lib version. Eg. "pstuil **cryptography==1.5 pyopenssl==16.0.0 ipython=2.2.0**" to avoid install error. This issue is related to Databricks and not related to AML SDK.
@@ -11,9 +11,9 @@ Kindly use this v4 notebooks (updated Sep 18)– if you had installed the AML SD
 The iPython Notebooks have to be run sequentially after making changes based on your subscription. The corresponding DBC archive contains all the notebooks and can be imported into your Databricks workspace. You can the run notebooks after importing .dbc instead of downloading individually.  
-This set of notebooks are related to Income prediction experiment based on this [dataset](https://archive.ics.uci.edu/ml/datasets/adult) and demonstrate how to data prep, train and operationalize a Spark ML model with Azure ML Python SDK from within Azure Databricks. For details on SDK concepts, please refer to [Private preview notebooks](https://github.com/Azure/ViennaDocs/tree/master/PrivatePreview/notebooks)
+This set of notebooks are related to Income prediction experiment based on this [dataset](https://archive.ics.uci.edu/ml/datasets/adult) and demonstrate how to data prep, train and operationalize a Spark ML model with Azure ML Python SDK from within Azure Databricks. For details on SDK concepts, please refer to [notebooks](https://github.com/Azure/MachineLearningNotebooks)
-(Recommended) [Azure Databricks AML SDK notebooks](Databricks_AMLSDK_github.dbc) A single DBC package to import all notebooks in your Databricks workspace.
+(Recommended) [Azure Databricks AML SDK notebooks](Databricks_AMLSDK_github.dbc) A single DBC package to import all notebooks in your Azure Databricks workspace.
 01. [Installation and Configuration](01.Installation_and_Configuration.ipynb): Install the Azure ML Python SDK and Initialize an Azure ML Workspace and save the Workspace configuration file.
 02. [Ingest data](02.Ingest_data.ipynb): Download the Adult Census Income dataset and split it into train and test sets.
@@ -23,4 +23,7 @@ This set of notebooks are related to Income prediction experiment based on this
 06. [Deploy to AKS](04.Deploy_to_AKS_existingImage.ipynb): Deploy model to Azure Kubernetis Service (AKS) with Azure ML Python SDK from an existing Image with model, conda and score file.
 Copyright (c) Microsoft Corporation. All rights reserved.
 All notebooks in this folder are licensed under the MIT License. 
 Apache®, Apache Spark, and Spark® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.
--- a/images/additems.png
+++ b/images/additems.png
--- a/onnx/README.md
+++ b/onnx/README.md
@@ -0,0 +1,23 @@
 # ONNX Runtime on Azure Machine Learning (AML)
 These tutorials show how to deploy pretrained [ONNX](http://onnx.ai) models on Azure virtual machines using [ONNX Runtime](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-build-deploy-onnx) for inference. By the end of the tutorial, you will deploy a state-of-the-art deep learning model on a virtual machine in Azure Machine Learning, using ONNX Runtime for Inference. You can ping the model with your own images to be analyzed!
 ## Tutorials
 - [Handwritten Digit Classification (MNIST) using ONNX Runtime on AzureML](https://github.com/Azure/MachineLearningNotebooks/blob/master/onnx/onnx-inference-mnist.ipynb)
 - [Facial Expression Recognition using ONNX Runtime on AzureML](https://github.com/Azure/MachineLearningNotebooks/blob/master/onnx/onnx-inference-emotion-recognition.ipynb)
 ## Documentation
 - [ONNX Runtime Python API Documentation](http://aka.ms/onnxruntime-python)
 - [Azure Machine Learning API Documentation](http://aka.ms/aml-docs)
 ## Related Articles
 - [Building and Deploying ONNX Runtime Models](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-build-deploy-onnx)
 - [Azure AI – Making AI Real for Business](https://aka.ms/aml-blog-overview) 
 - [What’s new in Azure Machine Learning](https://aka.ms/aml-blog-whats-new)
 ## License
 Copyright (c) Microsoft Corporation. All rights reserved.  
 Licensed under the MIT License.
--- a/onnx/onnx-inference-emotion-recognition.ipynb
+++ b/onnx/onnx-inference-emotion-recognition.ipynb
@@ -12,7 +12,7 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "# Facial Expression Recognition using ONNX Runtime on AzureML\n",
+    "# Facial Expression Recognition (Emotion FER+) using ONNX Runtime on Azure ML\n",
    "\n",
    "This example shows how to deploy an image classification neural network using the Facial Expression Recognition ([FER](https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data)) dataset and Open Neural Network eXchange format ([ONNX](http://aka.ms/onnxdocarticle)) on the Azure Machine Learning platform. This tutorial will show you how to deploy a FER+ model from the [ONNX model zoo](https://github.com/onnx/models), use it to make predictions using ONNX Runtime Inference, and deploy it as a web service in Azure.\n",
    "\n",
@@ -34,32 +34,54 @@
    "## Prerequisites\n",
    "\n",
    "### 1. Install Azure ML SDK and create a new workspace\n",
-    "Please follow [00.configuration.ipynb](https://github.com/Azure/MachineLearningNotebooks/blob/master/00.configuration.ipynb) notebook.\n",
+    "Please follow [Azure ML configuration notebook](https://github.com/Azure/MachineLearningNotebooks/blob/master/00.configuration.ipynb) to set up your environment.\n",
    "\n",
    "\n",
    "### 2. Install additional packages needed for this Notebook\n",
-    "You need to install the popular plotting library `matplotlib`, the image manipulation library `PIL`, and the `onnx` library in the conda environment where Azure Maching Learning SDK is installed.\n",
+    "You need to install the popular plotting library `matplotlib`, the image manipulation library `opencv`, and the `onnx` library in the conda environment where Azure Maching Learning SDK is installed.\n",
    "\n",
    "```sh\n",
-    "(myenv) $ pip install matplotlib onnx Pillow\n",
+    "(myenv) $ pip install matplotlib onnx opencv-python\n",
    "```\n",
    "\n",
    "**Debugging tip**: Make sure that to activate your virtual environment (myenv) before you re-launch this notebook using the `jupyter notebook` comand. Choose the respective Python kernel for your new virtual environment using the `Kernel > Change Kernel` menu above. If you have completed the steps correctly, the upper right corner of your screen should state `Python [conda env:myenv]` instead of `Python [default]`.\n",
    "\n",
    "### 3. Download sample data and pre-trained ONNX model from ONNX Model Zoo.\n",
    "\n",
-    "[Download the ONNX Emotion FER+ model and corresponding test data](https://www.cntk.ai/OnnxModels/emotion_ferplus/opset_7/emotion_ferplus.tar.gz) and place them in the same folder as this tutorial notebook. You can unzip the file through the following line of code.\n",
+    "In the following lines of code, we download [the trained ONNX Emotion FER+ model and corresponding test data](https://github.com/onnx/models/tree/master/emotion_ferplus) and place them in the same folder as this tutorial notebook. For more information about the FER+ dataset, please visit Microsoft Researcher Emad Barsoum's [FER+ source data repository](https://github.com/ebarsoum/FERPlus)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# urllib is a built-in Python library to download files from URLs\n",
    "\n",
-    "```sh\n",
+    "# Objective: retrieve the latest version of the ONNX Emotion FER+ model files from the\n",
-    "(myenv) $ tar xvzf emotion_ferplus.tar.gz\n",
+    "# ONNX Model Zoo and save it in the same folder as this tutorial\n",
    "```\n",
    "\n",
-    "More information can be found about the ONNX FER+ model on [github](https://github.com/onnx/models/tree/master/emotion_ferplus). For more information about the FER+ dataset, please visit Microsoft Researcher Emad Barsoum's [FER+ source data repository](https://github.com/ebarsoum/FERPlus)."
+    "import urllib.request\n",
    "\n",
    "onnx_model_url = \"https://www.cntk.ai/OnnxModels/emotion_ferplus/opset_7/emotion_ferplus.tar.gz\"\n",
    "\n",
    "urllib.request.urlretrieve(onnx_model_url, filename=\"emotion_ferplus.tar.gz\")\n",
    "\n",
    "# the ! magic command tells our jupyter notebook kernel to run the following line of \n",
    "# code from the command line instead of the notebook kernel\n",
    "\n",
    "# We use tar and xvcf to unzip the files we just retrieved from the ONNX model zoo\n",
    "\n",
    "!tar xvzf emotion_ferplus.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Load Azure ML workspace\n",
+    "## Deploy a VM with your ONNX model in the Cloud\n",
    "\n",
    "### Load Azure ML workspace\n",
    "\n",
    "We begin by instantiating a workspace object from the existing workspace created earlier in the configuration notebook."
   ]
@@ -147,9 +169,9 @@
   "source": [
    "### ONNX FER+ Model Methodology\n",
    "\n",
-    "The image classification model we are using is pre-trained using Microsoft's deep learning cognitive toolkit, [CNTK](https://github.com/Microsoft/CNTK), from the [ONNX model zoo](http://github.com/onnx/models). The model zoo has many other models that can be deployed on cloud providers like AzureML without any additional training. To ensure that our cloud deployed model works, we use testing data from the famous FER+ data set, provided as part of the [trained Emotion Recognition model](https://github.com/onnx/models/tree/master/emotion_ferplus) in the ONNX model zoo.\n",
+    "The image classification model we are using is pre-trained using Microsoft's deep learning cognitive toolkit, [CNTK](https://github.com/Microsoft/CNTK), from the [ONNX model zoo](http://github.com/onnx/models). The model zoo has many other models that can be deployed on cloud providers like AzureML without any additional training. To ensure that our cloud deployed model works, we use testing data from the well-known FER+ data set, provided as part of the [trained Emotion Recognition model](https://github.com/onnx/models/tree/master/emotion_ferplus) in the ONNX model zoo.\n",
    "\n",
-    "The original Facial Emotion Recognition (FER) Dataset was released in 2013, but some of the labels are not entirely appropriate for the expression. In the FER+ Dataset, each photo was evaluated by at least 10 croud sourced reviewers, creating a better basis for ground truth. \n",
+    "The original Facial Emotion Recognition (FER) Dataset was released in 2013 by Pierre-Luc Carrier and Aaron Courville as part of a [Kaggle Competition](https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data), but some of the labels are not entirely appropriate for the expression. In the FER+ Dataset, each photo was evaluated by at least 10 croud sourced reviewers, creating a more accurate basis for ground truth. \n",
    "\n",
    "You can see the difference of label quality in the sample model input below. The FER labels are the first word below each image, and the FER+ labels are the second word below each image.\n",
    "\n",
@@ -202,20 +224,18 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Deploy our model on Azure ML"
+    "### Specify our Score and Environment Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "We are now going to deploy our ONNX Model on AML with inference in ONNX Runtime. We begin by writing a score.py file, which will help us run the model in our Azure ML virtual machine (VM), and then specify our environment by writing a yml file.\n",
+    "We are now going to deploy our ONNX Model on AML with inference in ONNX Runtime. We begin by writing a score.py file, which will help us run the model in our Azure ML virtual machine (VM), and then specify our environment by writing a yml file. You will also notice that we import the onnxruntime library to do runtime inference on our ONNX models (passing in input and evaluating out model's predicted output). More information on the API and commands can be found in the [ONNX Runtime documentation](https://aka.ms/onnxruntime).\n",
    "\n",
    "You will also notice that we import the onnxruntime library to do runtime inference on our ONNX models (passing in input and evaluating out model's predicted output). More information on the API and commands can be found in the [ONNX Runtime documentation](https://aka.ms/onnxruntime).\n",
    "\n",
    "### Write Score File\n",
    "\n",
-    "A score file is what tells our Azure cloud service what to do. After initializing our model using azureml.core.model, we start an ONNX Runtime GPU inference session to evaluate the data passed in on our function calls."
+    "A score file is what tells our Azure cloud service what to do. After initializing our model using azureml.core.model, we start an ONNX Runtime inference session to evaluate the data passed in on our function calls."
   ]
  },
  {
@@ -248,10 +268,13 @@
    "    try:\n",
    "        # load in our data, convert to readable format\n",
    "        data = np.array(json.loads(input_data)['data']).astype('float32')\n",
    "        \n",
    "        start = time.time()\n",
    "        r = session.run([output_name], {input_name : data})\n",
    "        end = time.time()\n",
    "        \n",
    "        result = emotion_map(postprocess(r[0]))\n",
    "        \n",
    "        result_dict = {\"result\": result,\n",
    "                      \"time_in_sec\": [end - start]}\n",
    "    except Exception as e:\n",
@@ -260,9 +283,12 @@
    "    return json.dumps(result_dict)\n",
    "\n",
    "def emotion_map(classes, N=1):\n",
-    "    \"\"\"Take the most probable labels (output of postprocess) and returns the top N emotional labels that fit the picture.\"\"\"\n",
+    "    \"\"\"Take the most probable labels (output of postprocess) and returns the \n",
    "    top N emotional labels that fit the picture.\"\"\"\n",
    "    \n",
    "    emotion_table = {'neutral':0, 'happiness':1, 'surprise':2, 'sadness':3, \n",
    "                     'anger':4, 'disgust':5, 'fear':6, 'contempt':7}\n",
    "    \n",
    "    emotion_table = {'neutral':0, 'happiness':1, 'surprise':2, 'sadness':3, 'anger':4, 'disgust':5, 'fear':6, 'contempt':7}\n",
    "    emotion_keys = list(emotion_table.keys())\n",
    "    emotions = []\n",
    "    for i in range(N):\n",
@@ -276,8 +302,8 @@
    "    return e_x / e_x.sum(axis=0)\n",
    "\n",
    "def postprocess(scores):\n",
-    "    \"\"\"This function takes the scores generated by the network and returns the class IDs in decreasing \n",
+    "    \"\"\"This function takes the scores generated by the network and \n",
-    "    order of probability.\"\"\"\n",
+    "    returns the class IDs in decreasing order of probability.\"\"\"\n",
    "    prob = softmax(scores)\n",
    "    prob = np.squeeze(prob)\n",
    "    classes = np.argsort(prob)[::-1]\n",
@@ -329,7 +355,7 @@
    "image_config = ContainerImage.image_configuration(execution_script = \"score.py\",\n",
    "                                                  runtime = \"python\",\n",
    "                                                  conda_file = \"myenv.yml\",\n",
-    "                                                  description = \"test\",\n",
+    "                                                  description = \"Emotion ONNX Runtime container\",\n",
    "                                                  tags = {\"demo\": \"onnx\"})\n",
    "\n",
    "\n",
@@ -346,8 +372,6 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Debugging\n",
    "\n",
    "In case you need to debug your code, the next line of code accesses the log file."
   ]
  },
@@ -364,9 +388,9 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "We're all set! Let's get our model chugging.\n",
+    "We're all done specifying what we want our virtual machine to do. Let's configure and deploy our container image.\n",
    "\n",
-    "## Deploy the container image"
+    "### Deploy the container image"
   ]
  },
  {
@@ -439,23 +463,57 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "# Testing and Evaluation"
+    "## Testing and Evaluation\n",
-   ]
+    "\n",
-  },
+    "### Useful Helper Functions\n",
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Useful Helper Functions\n",
    "\n",
    "We preprocess and postprocess our data (see score.py file) using the helper functions specified in the [ONNX FER+ Model page in the Model Zoo repository](https://github.com/onnx/models/tree/master/emotion_ferplus)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def emotion_map(classes, N=1):\n",
    "    \"\"\"Take the most probable labels (output of postprocess) and returns the \n",
    "    top N emotional labels that fit the picture.\"\"\"\n",
    "    \n",
    "    emotion_table = {'neutral':0, 'happiness':1, 'surprise':2, 'sadness':3, \n",
    "                     'anger':4, 'disgust':5, 'fear':6, 'contempt':7}\n",
    "    \n",
    "    emotion_keys = list(emotion_table.keys())\n",
    "    emotions = []\n",
    "    for i in range(N):\n",
    "        emotions.append(emotion_keys[classes[i]])\n",
    "        \n",
    "    return emotions\n",
    "\n",
    "def softmax(x):\n",
    "    \"\"\"Compute softmax values (probabilities from 0 to 1) for each possible label.\"\"\"\n",
    "    x = x.reshape(-1)\n",
    "    e_x = np.exp(x - np.max(x))\n",
    "    return e_x / e_x.sum(axis=0)\n",
    "\n",
    "def postprocess(scores):\n",
    "    \"\"\"This function takes the scores generated by the network and \n",
    "    returns the class IDs in decreasing order of probability.\"\"\"\n",
    "    prob = softmax(scores)\n",
    "    prob = np.squeeze(prob)\n",
    "    classes = np.argsort(prob)[::-1]\n",
    "    return classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "### Load Test Data"
+    "### Load Test Data\n",
    "\n",
    "These are already in your directory from your ONNX model download (from the model zoo).\n",
    "\n",
    "Notice that our Model Zoo files have a .pb extension. This is because they are [protobuf files (Protocol Buffers)](https://developers.google.com/protocol-buffers/docs/pythontutorial), so we need to read in our data through our ONNX TensorProto reader into a format we can work with, like numerical arrays."
   ]
  },
  {
@@ -475,8 +533,6 @@
    "import json\n",
    "import os\n",
    "\n",
    "from score import emotion_map, softmax, postprocess\n",
    "\n",
    "test_inputs = []\n",
    "test_outputs = []\n",
    "\n",
@@ -512,7 +568,7 @@
   },
   "source": [
    "### Show some sample images\n",
-    "We use `matplotlib` to plot 3 test images from the model zoo with their labels over them."
+    "We use `matplotlib` to plot 3 test images from the dataset."
   ]
  },
  {
@@ -532,7 +588,7 @@
    "    plt.axhline('')\n",
    "    plt.axvline('')\n",
    "    plt.text(x = 10, y = -10, s = test_outputs[test_image], fontsize = 18)\n",
-    "    plt.imshow(test_inputs[test_image].reshape(64, 64), cmap = plt.cm.Greys)\n",
+    "    plt.imshow(test_inputs[test_image].reshape(64, 64), cmap = plt.cm.gray)\n",
    "plt.show()"
   ]
  },
@@ -571,7 +627,7 @@
    "        print(r['error'])\n",
    "        break\n",
    "        \n",
-    "    result = r['result'][0][0]\n",
+    "    result = r['result'][0]\n",
    "    time_ms = np.round(r['time_in_sec'][0] * 1000, 2)\n",
    "    \n",
    "    ground_truth = test_outputs[i]\n",
@@ -583,7 +639,7 @@
    "\n",
    "    # use different color for misclassified sample\n",
    "    font_color = 'red' if ground_truth != result else 'black'\n",
-    "    clr_map = plt.cm.gray if ground_truth != result else plt.cm.Greys\n",
+    "    clr_map = plt.cm.Greys if ground_truth != result else plt.cm.gray\n",
    "\n",
    "    # ground truth labels are in blue\n",
    "    plt.text(x = 10, y = -70, s = ground_truth, fontsize = 18, color = 'blue')\n",
@@ -611,15 +667,30 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "from PIL import Image\n",
+    "# Preprocessing functions take your image and format it so it can be passed\n",
    "# as input into our ONNX model\n",
    "\n",
-    "def preprocess(image_path):\n",
+    "import cv2\n",
-    "    input_shape = (1, 1, 64, 64)\n",
+    "\n",
-    "    img = Image.open(image_path)\n",
+    "def rgb2gray(rgb):\n",
-    "    img = img.resize((64, 64), Image.ANTIALIAS)\n",
+    "    \"\"\"Convert the input image into grayscale\"\"\"\n",
-    "    img_data = np.array(img)\n",
+    "    return np.dot(rgb[...,:3], [0.299, 0.587, 0.114])\n",
-    "    img_data = np.resize(img_data, input_shape)\n",
+    "\n",
-    "    return img_data"
+    "def resize_img(img):\n",
    "    \"\"\"Resize image to MNIST model input dimensions\"\"\"\n",
    "    img = cv2.resize(img, dsize=(64, 64), interpolation=cv2.INTER_AREA)\n",
    "    img.resize((1, 1, 64, 64))\n",
    "    return img\n",
    "\n",
    "def preprocess(img):\n",
    "    \"\"\"Resize input images and convert them to grayscale.\"\"\"\n",
    "    if img.shape == (64, 64):\n",
    "        img.resize((1, 1, 64, 64))\n",
    "        return img\n",
    "    \n",
    "    grayscale = rgb2gray(img)\n",
    "    processed_img = resize_img(grayscale)\n",
    "    return processed_img"
   ]
  },
  {
@@ -636,12 +707,15 @@
    "\n",
    "# e.g. your_test_image = \"C://Users//vinitra.swamy//Pictures//emotion_test_images//img_1.png\"\n",
    "\n",
-    "your_test_image = \"<path to file>\"\n",
+    "import matplotlib.image as mpimg\n",
    "\n",
    "if your_test_image != \"<path to file>\":\n",
-    "    img = preprocess(your_test_image)\n",
+    "    img = mpimg.imread(your_test_image)\n",
    "    plt.subplot(1,3,1)\n",
-    "    plt.imshow(img.reshape((64,64)), cmap = plt.cm.gray)\n",
+    "    plt.imshow(img, cmap = plt.cm.Greys)\n",
    "    print(\"Old Dimensions: \", img.shape)\n",
    "    img = preprocess(img)\n",
    "    print(\"New Dimensions: \", img.shape)\n",
    "else:\n",
    "    img = None"
   ]
@@ -659,7 +733,7 @@
    "\n",
    "    try:\n",
    "        r = json.loads(aci_service.run(input_data))\n",
-    "        result = r['result'][0][0]\n",
+    "        result = r['result'][0]\n",
    "        time_ms = np.round(r['time_in_sec'][0] * 1000, 2)\n",
    "    except Exception as e:\n",
    "        print(str(e))\n",
@@ -668,12 +742,13 @@
    "    plt.subplot(1,8,1)\n",
    "    plt.axhline('')\n",
    "    plt.axvline('')\n",
-    "    plt.text(x = -10, y = -35, s = \"Model prediction: \", fontsize = 14)\n",
+    "    plt.text(x = -10, y = -40, s = \"Model prediction: \", fontsize = 14)\n",
-    "    plt.text(x = -10, y = -20, s = \"Inference time: \", fontsize = 14)\n",
+    "    plt.text(x = -10, y = -25, s = \"Inference time: \", fontsize = 14)\n",
-    "    plt.text(x = 100, y = -35, s = str(result), fontsize = 14)\n",
+    "    plt.text(x = 100, y = -40, s = str(result), fontsize = 14)\n",
-    "    plt.text(x = 100, y = -20, s = str(time_ms) + \" ms\", fontsize = 14)\n",
+    "    plt.text(x = 100, y = -25, s = str(time_ms) + \" ms\", fontsize = 14)\n",
-    "    plt.text(x = -10, y = -8, s = \"Input image: \", fontsize = 14)\n",
+    "    plt.text(x = -10, y = -10, s = \"Model Input image: \", fontsize = 14)\n",
-    "    plt.imshow(img.reshape(64, 64), cmap = plt.cm.gray)    "
+    "    plt.imshow(img.reshape((64, 64)), cmap = plt.cm.gray)    \n",
    "     "
   ]
  },
  {
@@ -684,7 +759,7 @@
   "source": [
    "# remember to delete your service after you are done using it!\n",
    "\n",
-    "# aci_service.delete()"
+    "aci_service.delete()"
   ]
  },
  {
@@ -709,9 +784,9 @@
 ],
 "metadata": {
  "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python [conda env:myenv]",
   "language": "python",
-   "name": "python3"
+   "name": "conda-env-myenv-py"
  },
  "language_info": {
   "codemirror_mode": {
@@ -723,7 +798,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.6"
  },
  "msauthor": "vinitra.swamy"
 },
--- a/onnx/onnx-inference-mnist.ipynb
+++ b/onnx/onnx-inference-mnist.ipynb
@@ -12,7 +12,7 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "# Handwritten Digit Classification (MNIST) using ONNX Runtime on AzureML\n",
+    "# Handwritten Digit Classification (MNIST) using ONNX Runtime on Azure ML\n",
    "\n",
    "This example shows how to deploy an image classification neural network using the Modified National Institute of Standards and Technology ([MNIST](http://yann.lecun.com/exdb/mnist/)) dataset and Open Neural Network eXchange format ([ONNX](http://aka.ms/onnxdocarticle)) on the Azure Machine Learning platform. MNIST is a popular dataset consisting of 70,000 grayscale images. Each image is a handwritten digit of 28x28 pixels, representing number from 0 to 9. This tutorial will show you how to deploy a MNIST model from the [ONNX model zoo](https://github.com/onnx/models), use it to make predictions using ONNX Runtime Inference, and deploy it as a web service in Azure.\n",
    "\n",
@@ -22,9 +22,9 @@
    "\n",
    "#### Tutorial Objectives:\n",
    "\n",
-    "1. Describe the MNIST dataset and pretrained Convolutional Neural Net ONNX model, stored in the ONNX model zoo.\n",
+    "- Describe the MNIST dataset and pretrained Convolutional Neural Net ONNX model, stored in the ONNX model zoo.\n",
-    "2. Deploy and run the pretrained MNIST ONNX model on an Azure Machine Learning instance\n",
+    "- Deploy and run the pretrained MNIST ONNX model on an Azure Machine Learning instance\n",
-    "3. Predict labels for test set data points in the cloud using ONNX Runtime and Azure ML"
+    "- Predict labels for test set data points in the cloud using ONNX Runtime and Azure ML"
   ]
  },
  {
@@ -34,31 +34,61 @@
    "## Prerequisites\n",
    "\n",
    "### 1. Install Azure ML SDK and create a new workspace\n",
-    "Please follow [00.configuration.ipynb](https://github.com/Azure/MachineLearningNotebooks/blob/master/00.configuration.ipynb) notebook.\n",
+    "Please follow [Azure ML configuration notebook](https://github.com/Azure/MachineLearningNotebooks/blob/master/00.configuration.ipynb) to set up your environment.\n",
    "\n",
-    "### 2. Install additional packages needed for this Notebook\n",
+    "### 2. Install additional packages needed for this tutorial notebook\n",
-    "You need to install the popular plotting library `matplotlib`, the image manipulation library `opencv`, and the `onnx` library in the conda environment where Azure Maching Learning SDK is installed.\n",
+    "You need to install the popular plotting library `matplotlib`, the image manipulation library `opencv`, and the `onnx` library in the conda environment where Azure Maching Learning SDK is installed. \n",
    "\n",
    "```sh\n",
    "(myenv) $ pip install matplotlib onnx opencv-python\n",
    "```\n",
    "\n",
    "**Debugging tip**: Make sure that you run the \"jupyter notebook\" command to launch this notebook after activating your virtual environment. Choose the respective Python kernel for your new virtual environment using the `Kernel > Change Kernel` menu above. If you have completed the steps correctly, the upper right corner of your screen should state `Python [conda env:myenv]` instead of `Python [default]`.\n",
    "\n",
    "### 3. Download sample data and pre-trained ONNX model from ONNX Model Zoo.\n",
    "\n",
-    "[Download the ONNX MNIST model and corresponding test data](https://www.cntk.ai/OnnxModels/mnist/opset_7/mnist.tar.gz) and place them in the same folder as this tutorial notebook. You can unzip the file through the following line of code.\n",
+    "In the following lines of code, we download [the trained ONNX MNIST model and corresponding test data](https://github.com/onnx/models/tree/master/mnist) and place them in the same folder as this tutorial notebook. For more information about the MNIST dataset, please visit [Yan LeCun's website](http://yann.lecun.com/exdb/mnist/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# urllib is a built-in Python library to download files from URLs\n",
    "\n",
-    "```sh\n",
+    "# Objective: retrieve the latest version of the ONNX MNIST model files from the\n",
-    "(myenv) $ tar xvzf mnist.tar.gz\n",
+    "# ONNX Model Zoo and save it in the same folder as this tutorial\n",
    "```\n",
    "\n",
-    "More information can be found about the ONNX MNIST model on [github](https://github.com/onnx/models/tree/master/mnist). For more information about the MNIST dataset, please visit [Yan LeCun's website](http://yann.lecun.com/exdb/mnist/)."
+    "import urllib.request\n",
    "\n",
    "onnx_model_url = \"https://www.cntk.ai/OnnxModels/mnist/opset_7/mnist.tar.gz\"\n",
    "\n",
    "urllib.request.urlretrieve(onnx_model_url, filename=\"mnist.tar.gz\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the ! magic command tells our jupyter notebook kernel to run the following line of \n",
    "# code from the command line instead of the notebook kernel\n",
    "\n",
    "# We use tar and xvcf to unzip the files we just retrieved from the ONNX model zoo\n",
    "\n",
    "!tar xvzf mnist.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Load Azure ML workspace\n",
+    "## Deploy a VM with your ONNX model in the Cloud\n",
    "\n",
    "### Load Azure ML workspace\n",
    "\n",
    "We begin by instantiating a workspace object from the existing workspace created earlier in the configuration notebook."
   ]
@@ -113,11 +143,11 @@
   "source": [
    "from azureml.core.model import Model\n",
    "\n",
-    "model = Model.register(model_path = model_dir + \"//model.onnx\",\n",
+    "model = Model.register(workspace = ws,\n",
    "                       model_path = model_dir + \"/\" + \"model.onnx\",\n",
    "                       model_name = \"mnist_1\",\n",
    "                       tags = {\"onnx\": \"demo\"},\n",
-    "                       description = \"MNIST image classification CNN from ONNX Model Zoo\",\n",
+    "                       description = \"MNIST image classification CNN from ONNX Model Zoo\",)"
    "                       workspace = ws)"
   ]
  },
  {
@@ -188,16 +218,14 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "## Deploy our model on Azure ML"
+    "### Specify our Score and Environment Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "We are now going to deploy our ONNX Model on AML with inference in ONNX Runtime. We begin by writing a score.py file, which will help us run the model in our Azure ML virtual machine (VM), and then specify our environment by writing a yml file.\n",
+    "We are now going to deploy our ONNX Model on AML with inference in ONNX Runtime. We begin by writing a score.py file, which will help us run the model in our Azure ML virtual machine (VM), and then specify our environment by writing a yml file. You will also notice that we import the onnxruntime library to do runtime inference on our ONNX models (passing in input and evaluating out model's predicted output). More information on the API and commands can be found in the [ONNX Runtime documentation](https://aka.ms/onnxruntime).\n",
    "\n",
    "You will also notice that we import the onnxruntime library to do runtime inference on our ONNX models (passing in input and evaluating out model's predicted output). More information on the API and commands can be found in the [ONNX Runtime documentation](https://aka.ms/onnxruntime).\n",
    "\n",
    "### Write Score File\n",
    "\n",
@@ -248,7 +276,7 @@
    "    return json.dumps(result_dict)\n",
    "\n",
    "def choose_class(result_prob):\n",
-    "    \"\"\"We use argmax to determine the right label to choose from our output, after calling softmax on the 10 numbers we receive\"\"\"\n",
+    "    \"\"\"We use argmax to determine the right label to choose from our output\"\"\"\n",
    "    return int(np.argmax(result_prob, axis=0))"
   ]
  },
@@ -256,14 +284,9 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "### Write Environment File"
+    "### Write Environment File\n",
-   ]
+    "\n",
-  },
+    "This step creates a YAML environment file that specifies which dependencies we would like to see in our Linux Virtual Machine."
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This step creates a YAML file that specifies which dependencies we would like to see in our Linux Virtual Machine."
   ]
  },
  {
@@ -289,10 +312,19 @@
   "metadata": {},
   "source": [
    "### Create the Container Image\n",
    "\n",
    "This step will likely take a few minutes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.image import ContainerImage\n",
    "help(ContainerImage.image_configuration)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
@@ -304,8 +336,8 @@
    "image_config = ContainerImage.image_configuration(execution_script = \"score.py\",\n",
    "                                                  runtime = \"python\",\n",
    "                                                  conda_file = \"myenv.yml\",\n",
-    "                                                  description = \"test\",\n",
+    "                                                  description = \"MNIST ONNX Runtime container\",\n",
-    "                                                  tags = {\"demo\": \"onnx\"})                                                 )\n",
+    "                                                  tags = {\"demo\": \"onnx\"}) \n",
    "\n",
    "\n",
    "image = ContainerImage.create(name = \"onnxtest\",\n",
@@ -321,8 +353,6 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Debugging\n",
    "\n",
    "In case you need to debug your code, the next line of code accesses the log file."
   ]
  },
@@ -339,9 +369,9 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "We're all set! Let's get our model chugging.\n",
+    "We're all done specifying what we want our virtual machine to do. Let's configure and deploy our container image.\n",
    "\n",
-    "## Deploy the container image"
+    "### Deploy the container image"
   ]
  },
  {
@@ -373,7 +403,7 @@
   "source": [
    "from azureml.core.webservice import Webservice\n",
    "\n",
-    "aci_service_name = 'onnx-demo-mnist'\n",
+    "aci_service_name = 'onnx-demo-mnist20'\n",
    "print(\"Service\", aci_service_name)\n",
    "\n",
    "aci_service = Webservice.deploy_from_image(deployment_config = aciconfig,\n",
@@ -414,16 +444,13 @@
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "# Testing and Evaluation"
+    "## Testing and Evaluation\n",
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Test Data\n",
    "\n",
-    "These are already in your directory from your ONNX model download (from the model zoo). If you didn't place your model and test data in the same directory as this notebook, edit the \"model_dir\" filename below."
+    "### Load Test Data\n",
    "\n",
    "These are already in your directory from your ONNX model download (from the model zoo).\n",
    "\n",
    "Notice that our Model Zoo files have a .pb extension. This is because they are [protobuf files (Protocol Buffers)](https://developers.google.com/protocol-buffers/docs/pythontutorial), so we need to read in our data through our ONNX TensorProto reader into a format we can work with, like numerical arrays."
   ]
  },
  {
@@ -579,7 +606,9 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "# Preprocessing functions\n",
+    "# Preprocessing functions take your image and format it so it can be passed\n",
    "# as input into our ONNX model\n",
    "\n",
    "import cv2\n",
    "\n",
    "def rgb2gray(rgb):\n",
@@ -587,12 +616,17 @@
    "    return np.dot(rgb[...,:3], [0.299, 0.587, 0.114])\n",
    "\n",
    "def resize_img(img):\n",
    "    \"\"\"Resize image to MNIST model input dimensions\"\"\"\n",
    "    img = cv2.resize(img, dsize=(28, 28), interpolation=cv2.INTER_AREA)\n",
    "    img.resize((1, 1, 28, 28))\n",
    "    return img\n",
    "\n",
    "def preprocess(img):\n",
    "    \"\"\"Resize input images and convert them to grayscale.\"\"\"\n",
    "    if img.shape == (28, 28):\n",
    "        img.resize((1, 1, 28, 28))\n",
    "        return img\n",
    "    \n",
    "    grayscale = rgb2gray(img)\n",
    "    processed_img = resize_img(grayscale)\n",
    "    return processed_img"
@@ -608,11 +642,8 @@
    "# Make sure your image is square and the dimensions are equal (i.e. 100 * 100 pixels or 28 * 28 pixels)\n",
    "\n",
    "# Any PNG or JPG image file should work\n",
    "# Make sure to include the entire path with // instead of /\n",
    "\n",
-    "# e.g. your_test_image = \"C://Users//vinitra.swamy//Pictures//digit.png\"\n",
+    "# e.g. your_test_image = \"C:/Users/vinitra.swamy/Pictures/handwritten_digit.png\"\n",
    "\n",
    "your_test_image = \"<path to file>\"\n",
    "\n",
    "import matplotlib.image as mpimg\n",
    "\n",
@@ -721,7 +752,7 @@
   "source": [
    "# remember to delete your service after you are done using it!\n",
    "\n",
-    "# aci_service.delete()"
+    "aci_service.delete()"
   ]
  },
  {
@@ -738,16 +769,16 @@
    "- ensured that your deep learning model is working perfectly (in the cloud) on test data, and checked it against some of your own!\n",
    "\n",
    "Next steps:\n",
-    "- Check out another interesting application based on a Microsoft Research computer vision paper that lets you set up a [facial emotion recognition model](https://github.com/Azure/MachineLearningNotebooks/tree/master/onnx/onnx-inference-emotion-recognition.ipynb) in the cloud! This tutorial deploys a pre-trained ONNX Computer Vision model in an Azure ML virtual machine with GPU support.\n",
+    "- Check out another interesting application based on a Microsoft Research computer vision paper that lets you set up a [facial emotion recognition model](https://github.com/Azure/MachineLearningNotebooks/tree/master/onnx/onnx-inference-emotion-recognition.ipynb) in the cloud! This tutorial deploys a pre-trained ONNX Computer Vision model in an Azure ML virtual machine.\n",
    "- Contribute to our [open source ONNX repository on github](http://github.com/onnx/onnx) and/or add to our [ONNX model zoo](http://github.com/onnx/models)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python [conda env:myenv]",
   "language": "python",
-   "name": "python3"
+   "name": "conda-env-myenv-py"
  },
  "language_info": {
   "codemirror_mode": {
@@ -759,7 +790,7 @@
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.6"
  },
  "msauthor": "vinitra.swamy"
 },
--- a/tutorials/02.deploy-models.ipynb
+++ b/tutorials/02.deploy-models.ipynb
@@ -39,7 +39,11 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "register model from file"
    ]
   },
   "outputs": [],
   "source": [
    "# If you did NOT complete the tutorial, you can instead run this cell \n",
@@ -86,7 +90,11 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "check version"
    ]
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
@@ -113,7 +121,12 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "load workspace",
     "download model"
    ]
   },
   "outputs": [],
   "source": [
    "from azureml.core import Workspace\n",
@@ -298,13 +311,18 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "set conda dependencies"
    ]
   },
   "outputs": [],
   "source": [
    "from azureml.core.conda_dependencies import CondaDependencies \n",
    "\n",
    "myenv = CondaDependencies()\n",
    "myenv.add_conda_package(\"scikit-learn\")\n",
    "myenv.add_pip_package(\"pynacl==1.2.1\")\n",
    "\n",
    "with open(\"myenv.yml\",\"w\") as f:\n",
    "    f.write(myenv.serialize_to_string())"
@@ -339,7 +357,12 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "configure web service",
     "aci"
    ]
   },
   "outputs": [],
   "source": [
    "from azureml.core.webservice import AciWebservice\n",
@@ -372,7 +395,14 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "configure image",
     "create image",
     "deploy web service",
     "aci"
    ]
   },
   "outputs": [],
   "source": [
    "%%time\n",
@@ -403,7 +433,11 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "get scoring uri"
    ]
   },
   "outputs": [],
   "source": [
    "print(service.scoring_uri)"
@@ -430,7 +464,11 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "score web service"
    ]
   },
   "outputs": [],
   "source": [
    "import json\n",
@@ -475,7 +513,11 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "score web service"
    ]
   },
   "outputs": [],
   "source": [
    "import requests\n",
@@ -511,7 +553,11 @@
  {
   "cell_type": "code",
   "execution_count": null,
-   "metadata": {},
+   "metadata": {
    "tags": [
     "delete web service"
    ]
   },
   "outputs": [],
   "source": [
    "service.delete()"
--- a/tutorials/03.auto-train-models.ipynb
+++ b/tutorials/03.auto-train-models.ipynb
@@ -133,8 +133,8 @@
    "digits = datasets.load_digits()\n",
    "\n",
    "# only take the first 100 rows if you want the training steps to run faster\n",
-    "X_digits = digits.data[100:,:]\n",
+    "X_digits = digits.data[:100,:]\n",
-    "y_digits = digits.target[100:]\n",
+    "y_digits = digits.target[:100]\n",
    "\n",
    "# use full dataset\n",
    "#X_digits = digits.data\n",
@@ -374,7 +374,7 @@
    "> * Review training results\n",
    "> * Register the best model\n",
    "\n",
-    "Learn more about [how to configure settings for automatic training]() or [how to use automatic training on a remote resource]()."
+    "Learn more about [how to configure settings for automatic training](https://aka.ms/aml-how-configure-auto) or [how to use automatic training on a remote resource](https://aka.ms/aml-how-to-auto-remote)."
   ]
  }
 ],
Author	SHA1	Message	Date
Sheri Gilley	b0b0756aed	fix links	2018-09-28 17:24:37 -05:00
Roope Astala	ff19151d0a	Merge pull request #29 from rastala/master onnx update	2018-09-28 16:42:09 -04:00
rastala	933c1ffc4e	onnx update	2018-09-28 16:41:21 -04:00
Roope Astala	f75faaa31e	Merge pull request #28 from rastala/master mitigation to image creation issue	2018-09-27 12:52:47 -04:00
Roope Astala	ae8874ad32	mitigation to image creation issue	2018-09-27 12:50:38 -04:00
Hai Ning	6c3abe2d03	Update train.py	2018-09-27 11:30:22 -04:00
Hai Ning	4627080ff4	Update 04.train-on-remote-vm.ipynb	2018-09-27 11:30:01 -04:00
Sheri Gilley	69af6e36fe	Merge pull request #23 from Azure/sdg-update update readme	2018-09-26 17:57:57 -05:00
Hai Ning	e27ab9a58e	Update 04.train-on-remote-vm.ipynb	2018-09-26 14:02:27 -04:00
Hai Ning	c85e7e52af	Update 04.train-on-remote-vm.ipynb	2018-09-26 14:01:39 -04:00
Hai Ning	5598e07729	Update 05.train-in-spark.ipynb	2018-09-26 14:00:38 -04:00
Roope Astala	d9b62ad651	Merge pull request #26 from rastala/master Updating Azure Databricks examples	2018-09-26 09:34:40 -04:00
Roope Astala	8aa287dadf	Updating Azure Databricks examples	2018-09-26 09:32:24 -04:00
Roope Astala	9ab092a4d0	Merge pull request #21 from sitomani/master Fixed an error on Data Exploration chapter	2018-09-25 19:14:30 -04:00
Sheri Gilley	1a1a81621f	remove file	2018-09-25 16:24:00 -05:00
Sheri Gilley	d93daa3f38	create link for nb	2018-09-25 16:21:14 -05:00
Sheri Gilley	2fb910b0e0	fix link	2018-09-25 16:17:34 -05:00
Sheri Gilley	2879e00884	update	2018-09-25 16:09:01 -05:00
Sheri Gilley	b574bfd3cf	updates	2018-09-25 16:03:57 -05:00
Sheri Gilley	6a3b814394	Update README.md	2018-09-25 15:07:24 -05:00
Sheri Gilley	1009ffab36	Update README.md	2018-09-25 15:06:20 -05:00
Sheri Gilley	995fb1ac8c	Update README.md	2018-09-25 14:46:25 -05:00
Sheri Gilley	e418e4fbb2	Update README.md	2018-09-25 14:46:01 -05:00
Sheri Gilley	cdbfa203e1	Update README.md	2018-09-25 14:36:26 -05:00
Roope Astala	a9a9635e72	Merge pull request #22 from rastala/master Update 00, 01 and 10 notebooks	2018-09-25 15:02:28 -04:00
Roope Astala	b568dc364f	Update 00, 01 and 10 notebooks	2018-09-25 15:01:08 -04:00
Aleksi Sitomaniemi	59bdd5a858	Fixed an error on Data Exploration chapter The sample code comment discusses using a portion of the full dataset to make training faster, suggesting that the following code takes 100 first samples from the dataset. However, the code in repository actually leaves the first 100 items out, and picks the rest of the full set into the evaluated X_digits, Y_digits matrices.	2018-09-25 12:28:51 +03:00