update samples from Release-68 as a part of SDK release

update samples from Release-67 as a part of  SDK release

README.md

@@ -65,7 +65,7 @@ Visit following repos to see projects contributed by Azure ML users:
  - [UMass Amherst Student Samples](https://github.com/katiehouse3/microsoft-azure-ml-notebooks) - A number of end-to-end machine learning notebooks, including machine translation, image classification, and customer churn, created by students in the 696DS course at UMass Amherst.
  
 ## Data/Telemetry 
-This repository collects usage data and sends it to Mircosoft to help improve our products and services. Read Microsoft's [privacy statement to learn more](https://privacy.microsoft.com/en-US/privacystatement)
+This repository collects usage data and sends it to Microsoft to help improve our products and services. Read Microsoft's [privacy statement to learn more](https://privacy.microsoft.com/en-US/privacystatement)
 
 To opt out of tracking, please go to the raw markdown or .ipynb files and remove the following line of code:
 

configuration.ipynb

@@ -103,7 +103,7 @@
       "source": [
         "import azureml.core\n",
         "\n",
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/README.md

@@ -4,7 +4,7 @@ Learn how to use Azure Machine Learning services for experimentation and model m
 
 As a pre-requisite, run the [configuration Notebook](../configuration.ipynb) notebook first to set up your Azure ML Workspace. Then, run the notebooks in following recommended order.
 
-* [train-within-notebook](./training/train-within-notebook): Train a model hile tracking run history, and learn how to deploy the model as web service to Azure Container Instance.
+* [train-within-notebook](./training/train-within-notebook): Train a model while tracking run history, and learn how to deploy the model as web service to Azure Container Instance.
 * [train-on-local](./training/train-on-local): Learn how to submit a run to local computer and use Azure ML managed run configuration.
 * [train-on-amlcompute](./training/train-on-amlcompute): Use a 1-n node Azure ML managed compute cluster for remote runs on Azure CPU or GPU infrastructure.
 * [train-on-remote-vm](./training/train-on-remote-vm): Use Data Science Virtual Machine as a target for remote runs.

how-to-use-azureml/automated-machine-learning/README.md

@@ -154,12 +154,6 @@ jupyter notebook
 - [auto-ml-continuous-retraining.ipynb](continuous-retraining/auto-ml-continuous-retraining.ipynb)
     - Continuous retraining using Pipelines and Time-Series TabularDataset
 
-- [auto-ml-classification-text-dnn.ipynb](classification-text-dnn/auto-ml-classification-text-dnn.ipynb)
-    - Classification with text data using deep learning in AutoML
-    - AutoML highlights here include using deep neural networks (DNNs) to create embedded features from text data.
-    - Depending on the compute cluster the user provides, AutoML tried out Bidirectional Encoder Representations from Transformers (BERT) when a GPU compute is used.
-    - Bidirectional Long-Short Term neural network (BiLSTM) when a CPU compute is used, thereby optimizing the choice of DNN for the uesr's setup.
-
 <a name="documentation"></a>
 See [Configure automated machine learning experiments](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-auto-train) to learn how more about the the settings and features available for automated machine learning experiments.
 

how-to-use-azureml/automated-machine-learning/automl_env.yml

@@ -6,12 +6,12 @@ dependencies:
 - python>=3.5.2,<3.6.8
 - nb_conda
 - matplotlib==2.1.0
-- numpy~=1.16.0
+- numpy~=1.18.0
 - cython
 - urllib3<1.24
 - scipy==1.4.1
-- scikit-learn>=0.19.0,<=0.20.3
-- pandas>=0.22.0,<=0.23.4
+- scikit-learn==0.22.1
+- pandas==0.25.1
 - py-xgboost<=0.90
 - conda-forge::fbprophet==0.5
 - holidays==0.9.11
@@ -20,12 +20,9 @@ dependencies:
 
 - pip:
   # Required packages for AzureML execution, history, and data preparation.
-  - azureml-defaults
-  - azureml-train-automl
-  - azureml-train
   - azureml-widgets
-  - azureml-pipeline
   - pytorch-transformers==1.0.0
   - spacy==2.1.8
   - https://aka.ms/automl-resources/packages/en_core_web_sm-2.1.0.tar.gz
+  - -r https://automlcesdkdataresources.blob.core.windows.net/validated-requirements/1.14.0/validated_win32_requirements.txt [--no-deps]
 

how-to-use-azureml/automated-machine-learning/automl_env_linux.yml

@@ -0,0 +1,28 @@
+name: azure_automl
+dependencies:
+  # The python interpreter version.
+  # Currently Azure ML only supports 3.5.2 and later.
+- pip<=19.3.1
+- python>=3.5.2,<3.6.8
+- nb_conda
+- matplotlib==2.1.0
+- numpy~=1.18.0
+- cython
+- urllib3<1.24
+- scipy==1.4.1
+- scikit-learn==0.22.1
+- pandas==0.25.1
+- py-xgboost<=0.90
+- conda-forge::fbprophet==0.5
+- holidays==0.9.11
+- pytorch::pytorch=1.4.0
+- cudatoolkit=10.1.243
+
+- pip:
+  # Required packages for AzureML execution, history, and data preparation.
+  - azureml-widgets
+  - pytorch-transformers==1.0.0
+  - spacy==2.1.8
+  - https://aka.ms/automl-resources/packages/en_core_web_sm-2.1.0.tar.gz
+  - -r https://automlcesdkdataresources.blob.core.windows.net/validated-requirements/1.14.0/validated_linux_requirements.txt [--no-deps]
+

how-to-use-azureml/automated-machine-learning/automl_env_mac.yml

@@ -7,12 +7,12 @@ dependencies:
 - python>=3.5.2,<3.6.8
 - nb_conda
 - matplotlib==2.1.0
-- numpy~=1.16.0
+- numpy~=1.18.0
 - cython
 - urllib3<1.24
 - scipy==1.4.1
-- scikit-learn>=0.19.0,<=0.20.3
-- pandas>=0.22.0,<=0.23.4
+- scikit-learn==0.22.1
+- pandas==0.25.1
 - py-xgboost<=0.90
 - conda-forge::fbprophet==0.5
 - holidays==0.9.11
@@ -21,11 +21,8 @@ dependencies:
 
 - pip:
   # Required packages for AzureML execution, history, and data preparation.
-  - azureml-defaults
-  - azureml-train-automl
-  - azureml-train
   - azureml-widgets
-  - azureml-pipeline
   - pytorch-transformers==1.0.0
   - spacy==2.1.8
   - https://aka.ms/automl-resources/packages/en_core_web_sm-2.1.0.tar.gz  
+  - -r https://automlcesdkdataresources.blob.core.windows.net/validated-requirements/1.14.0/validated_darwin_requirements.txt [--no-deps]

how-to-use-azureml/automated-machine-learning/automl_setup_linux.sh

@@ -12,7 +12,7 @@ fi
 
 if [ "$AUTOML_ENV_FILE" == "" ]
 then
-  AUTOML_ENV_FILE="automl_env.yml"
+  AUTOML_ENV_FILE="automl_env_linux.yml"
 fi
 
 if [ ! -f $AUTOML_ENV_FILE ]; then

how-to-use-azureml/automated-machine-learning/classification-bank-marketing-all-features/auto-ml-classification-bank-marketing-all-features.ipynb

@@ -105,7 +105,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -500,11 +500,10 @@
       "source": [
         "# Wait for the best model explanation run to complete\n",
         "from azureml.core.run import Run\n",
-        "model_explainability_run_id = remote_run.get_properties().get('ModelExplainRunId')\n",
+        "model_explainability_run_id = remote_run.id + \"_\" + \"ModelExplain\"\n",
         "print(model_explainability_run_id)\n",
-        "if model_explainability_run_id is not None:\n",
-        "    model_explainability_run = Run(experiment=experiment, run_id=model_explainability_run_id)\n",
-        "    model_explainability_run.wait_for_completion()\n",
+        "model_explainability_run = Run(experiment=experiment, run_id=model_explainability_run_id)\n",
+        "model_explainability_run.wait_for_completion()\n",
         "\n",
         "# Get the best run object\n",
         "best_run, fitted_model = remote_run.get_output()"

how-to-use-azureml/automated-machine-learning/classification-credit-card-fraud/auto-ml-classification-credit-card-fraud.ipynb

@@ -93,7 +93,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/classification-text-dnn/auto-ml-classification-text-dnn.ipynb

@@ -1,587 +0,0 @@
-{
-  "cells": [
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
-        "\n",
-        "Licensed under the MIT License."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/automated-machine-learning/classification-text-dnn/auto-ml-classification-text-dnn.png)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "# Automated Machine Learning\n",
-        "_**Text Classification Using Deep Learning**_\n",
-        "\n",
-        "## Contents\n",
-        "1. [Introduction](#Introduction)\n",
-        "1. [Setup](#Setup)\n",
-        "1. [Data](#Data)\n",
-        "1. [Train](#Train)\n",
-        "1. [Evaluate](#Evaluate)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Introduction\n",
-        "This notebook demonstrates classification with text data using deep learning in AutoML.\n",
-        "\n",
-        "AutoML highlights here include using deep neural networks (DNNs) to create embedded features from text data. Depending on the compute cluster the user provides, AutoML tried out Bidirectional Encoder Representations from Transformers (BERT) when a GPU compute is used, and Bidirectional Long-Short Term neural network (BiLSTM) when a CPU compute is used, thereby optimizing the choice of DNN for the uesr's setup.\n",
-        "\n",
-        "Make sure you have executed the [configuration](../../../configuration.ipynb) before running this notebook.\n",
-        "\n",
-        "An Enterprise workspace is required for this notebook. To learn more about creating an Enterprise workspace or upgrading to an Enterprise workspace from the Azure portal, please visit our [Workspace page](https://docs.microsoft.com/azure/machine-learning/service/concept-workspace#upgrade).\n",
-        "\n",
-        "Notebook synopsis:\n",
-        "1. Creating an Experiment in an existing Workspace\n",
-        "2. Configuration and remote run of AutoML for a text dataset (20 Newsgroups dataset from scikit-learn) for classification\n",
-        "3. Registering the best model for future use\n",
-        "4. Evaluating the final model on a test set"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Setup"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "import logging\n",
-        "import os\n",
-        "import shutil\n",
-        "\n",
-        "import pandas as pd\n",
-        "\n",
-        "import azureml.core\n",
-        "from azureml.core.experiment import Experiment\n",
-        "from azureml.core.workspace import Workspace\n",
-        "from azureml.core.dataset import Dataset\n",
-        "from azureml.core.compute import AmlCompute\n",
-        "from azureml.core.compute import ComputeTarget\n",
-        "from azureml.core.run import Run\n",
-        "from azureml.widgets import RunDetails\n",
-        "from azureml.core.model import Model \n",
-        "from helper import run_inference, get_result_df\n",
-        "from azureml.train.automl import AutoMLConfig\n",
-        "from sklearn.datasets import fetch_20newsgroups"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "This sample notebook may use features that are not available in previous versions of the Azure ML SDK."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
-        "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "As part of the setup you have already created a <b>Workspace</b>. To run AutoML, you also need to create an <b>Experiment</b>. An Experiment corresponds to a prediction problem you are trying to solve, while a Run corresponds to a specific approach to the problem."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "ws = Workspace.from_config()\n",
-        "\n",
-        "# Choose an experiment name.\n",
-        "experiment_name = 'automl-classification-text-dnn'\n",
-        "\n",
-        "experiment = Experiment(ws, experiment_name)\n",
-        "\n",
-        "output = {}\n",
-        "output['Subscription ID'] = ws.subscription_id\n",
-        "output['Workspace Name'] = ws.name\n",
-        "output['Resource Group'] = ws.resource_group\n",
-        "output['Location'] = ws.location\n",
-        "output['Experiment Name'] = experiment.name\n",
-        "pd.set_option('display.max_colwidth', -1)\n",
-        "outputDf = pd.DataFrame(data = output, index = [''])\n",
-        "outputDf.T"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Set up a compute cluster\n",
-        "This section uses a user-provided compute cluster (named \"dnntext-cluster\" in this example). If a cluster with this name does not exist in the user's workspace, the below code will create a new cluster. You can choose the parameters of the cluster as mentioned in the comments.\n",
-        "\n",
-        "Whether you provide/select a CPU or GPU cluster, AutoML will choose the appropriate DNN for that setup - BiLSTM or BERT text featurizer will be included in the candidate featurizers on CPU and GPU respectively.  If your goal is to obtain the most accurate model, we recommend you use GPU clusters since BERT featurizers usually outperform BiLSTM featurizers."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.compute import ComputeTarget, AmlCompute\n",
-        "from azureml.core.compute_target import ComputeTargetException\n",
-        "\n",
-        "# Choose a name for your cluster.\n",
-        "amlcompute_cluster_name = \"dnntext-cluster\"\n",
-        "\n",
-        "# Verify that cluster does not exist already\n",
-        "try:\n",
-        "    compute_target = ComputeTarget(workspace=ws, name=amlcompute_cluster_name)\n",
-        "    print('Found existing cluster, use it.')\n",
-        "except ComputeTargetException:\n",
-        "    compute_config = AmlCompute.provisioning_configuration(vm_size = \"STANDARD_NC6\", # CPU for BiLSTM, such as \"STANDARD_D2_V2\" \n",
-        "                                                           # To use BERT (this is recommended for best performance), select a GPU such as \"STANDARD_NC6\" \n",
-        "                                                           # or similar GPU option\n",
-        "                                                           # available in your workspace\n",
-        "                                                           max_nodes = 1)\n",
-        "    compute_target = ComputeTarget.create(ws, amlcompute_cluster_name, compute_config)\n",
-        "\n",
-        "compute_target.wait_for_completion(show_output=True)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "### Get data\n",
-        "For this notebook we will use 20 Newsgroups data from scikit-learn. We filter the data to contain four classes and take a sample as training data. Please note that for accuracy improvement, more data is needed. For this notebook we provide a small-data example so that you can use this template to use with your larger sized data."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "data_dir = \"text-dnn-data\" # Local directory to store data\n",
-        "blobstore_datadir = data_dir # Blob store directory to store data in\n",
-        "target_column_name = 'y'\n",
-        "feature_column_name = 'X'\n",
-        "\n",
-        "def get_20newsgroups_data():\n",
-        "    '''Fetches 20 Newsgroups data from scikit-learn\n",
-        "       Returns them in form of pandas dataframes\n",
-        "    '''\n",
-        "    remove = ('headers', 'footers', 'quotes')\n",
-        "    categories = [\n",
-        "        'rec.sport.baseball',\n",
-        "        'rec.sport.hockey',\n",
-        "        'comp.graphics',\n",
-        "        'sci.space',\n",
-        "        ]\n",
-        "\n",
-        "    data = fetch_20newsgroups(subset = 'train', categories = categories,\n",
-        "                                    shuffle = True, random_state = 42,\n",
-        "                                    remove = remove)\n",
-        "    data = pd.DataFrame({feature_column_name: data.data, target_column_name: data.target})\n",
-        "\n",
-        "    data_train = data[:200]\n",
-        "    data_test = data[200:300]    \n",
-        "\n",
-        "    data_train = remove_blanks_20news(data_train, feature_column_name, target_column_name)\n",
-        "    data_test = remove_blanks_20news(data_test, feature_column_name, target_column_name)\n",
-        "    \n",
-        "    return data_train, data_test\n",
-        "    \n",
-        "def remove_blanks_20news(data, feature_column_name, target_column_name):\n",
-        "    \n",
-        "    data[feature_column_name] = data[feature_column_name].replace(r'\\n', ' ', regex=True).apply(lambda x: x.strip())\n",
-        "    data = data[data[feature_column_name] != '']\n",
-        "    \n",
-        "    return data"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "#### Fetch data and upload to datastore for use in training"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "data_train, data_test = get_20newsgroups_data()\n",
-        "\n",
-        "if not os.path.isdir(data_dir):\n",
-        "    os.mkdir(data_dir)\n",
-        "    \n",
-        "train_data_fname = data_dir + '/train_data.csv'\n",
-        "test_data_fname = data_dir + '/test_data.csv'\n",
-        "\n",
-        "data_train.to_csv(train_data_fname, index=False)\n",
-        "data_test.to_csv(test_data_fname, index=False)\n",
-        "\n",
-        "datastore = ws.get_default_datastore()\n",
-        "datastore.upload(src_dir=data_dir, target_path=blobstore_datadir,\n",
-        "                    overwrite=True)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "train_dataset = Dataset.Tabular.from_delimited_files(path = [(datastore, blobstore_datadir + '/train_data.csv')])"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "### Prepare AutoML run"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "This step requires an Enterprise workspace to gain access to this feature. To learn more about creating an Enterprise workspace or upgrading to an Enterprise workspace from the Azure portal, please visit our [Workspace page](https://docs.microsoft.com/azure/machine-learning/service/concept-workspace#upgrade)."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "automl_settings = {\n",
-        "    \"experiment_timeout_minutes\": 20,\n",
-        "    \"primary_metric\": 'accuracy',\n",
-        "    \"max_concurrent_iterations\": 4, \n",
-        "    \"max_cores_per_iteration\": -1,\n",
-        "    \"enable_dnn\": True,\n",
-        "    \"enable_early_stopping\": True,\n",
-        "    \"validation_size\": 0.3,\n",
-        "    \"verbosity\": logging.INFO,\n",
-        "    \"enable_voting_ensemble\": False,\n",
-        "    \"enable_stack_ensemble\": False,\n",
-        "}\n",
-        "\n",
-        "automl_config = AutoMLConfig(task = 'classification',\n",
-        "                             debug_log = 'automl_errors.log',\n",
-        "                             compute_target=compute_target,\n",
-        "                             training_data=train_dataset,\n",
-        "                             label_column_name=target_column_name,\n",
-        "                             **automl_settings\n",
-        "                            )"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "#### Submit AutoML Run"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "automl_run = experiment.submit(automl_config, show_output=True)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "automl_run"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Displaying the run objects gives you links to the visual tools in the Azure Portal. Go try them!"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "### Retrieve the Best Model\n",
-        "Below we select the best model pipeline from our iterations, use it to test on test data on the same compute cluster."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "You can test the model locally to get a feel of the input/output. When the model contains BERT, this step will require pytorch and pytorch-transformers installed in your local environment. The exact versions of these packages can be found in the **automl_env.yml** file located in the local copy of your MachineLearningNotebooks folder here:\n",
-        "MachineLearningNotebooks/how-to-use-azureml/automated-machine-learning/automl_env.yml"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "best_run, fitted_model = automl_run.get_output()"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "You can now see what text transformations are used to convert text data to features for this dataset, including deep learning transformations based on BiLSTM or Transformer (BERT is one implementation of a Transformer) models."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "text_transformations_used = []\n",
-        "for column_group in fitted_model.named_steps['datatransformer'].get_featurization_summary():\n",
-        "    text_transformations_used.extend(column_group['Transformations'])\n",
-        "text_transformations_used"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "### Registering the best model\n",
-        "We now register the best fitted model from the AutoML Run for use in future deployments.  "
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Get results stats, extract the best model from AutoML run, download and register the resultant best model"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "summary_df = get_result_df(automl_run)\n",
-        "best_dnn_run_id = summary_df['run_id'].iloc[0]\n",
-        "best_dnn_run = Run(experiment, best_dnn_run_id)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "model_dir = 'Model' # Local folder where the model will be stored temporarily\n",
-        "if not os.path.isdir(model_dir):\n",
-        "    os.mkdir(model_dir)\n",
-        "    \n",
-        "best_dnn_run.download_file('outputs/model.pkl', model_dir + '/model.pkl')"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Register the model in your Azure Machine Learning Workspace. If you previously registered a model, please make sure to delete it so as to replace it with this new model."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# Register the model\n",
-        "model_name = 'textDNN-20News'\n",
-        "model = Model.register(model_path = model_dir + '/model.pkl',\n",
-        "                       model_name = model_name,\n",
-        "                       tags=None,\n",
-        "                       workspace=ws)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Evaluate on Test Data"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "We now use the best fitted model from the AutoML Run to make predictions on the test set.  \n",
-        "\n",
-        "Test set schema should match that of the training set."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "test_dataset = Dataset.Tabular.from_delimited_files(path = [(datastore, blobstore_datadir + '/test_data.csv')])\n",
-        "\n",
-        "# preview the first 3 rows of the dataset\n",
-        "test_dataset.take(3).to_pandas_dataframe()"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "test_experiment = Experiment(ws, experiment_name + \"_test\")"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "script_folder = os.path.join(os.getcwd(), 'inference')\n",
-        "os.makedirs(script_folder, exist_ok=True)\n",
-        "shutil.copy('infer.py', script_folder)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "test_run = run_inference(test_experiment, compute_target, script_folder, best_dnn_run,\n",
-        "                         train_dataset, test_dataset, target_column_name, model_name)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Display computed metrics"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "test_run"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "RunDetails(test_run).show()"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "test_run.wait_for_completion()"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "pd.Series(test_run.get_metrics())"
-      ]
-    }
-  ],
-  "metadata": {
-    "authors": [
-      {
-        "name": "anshirga"
-      }
-    ],
-    "compute": [
-      "AML Compute"
-    ],
-    "datasets": [
-      "None"
-    ],
-    "deployment": [
-      "None"
-    ],
-    "exclude_from_index": false,
-    "framework": [
-      "None"
-    ],
-    "friendly_name": "DNN Text Featurization",
-    "index_order": 2,
-    "kernelspec": {
-      "display_name": "Python 3.6",
-      "language": "python",
-      "name": "python36"
-    },
-    "language_info": {
-      "codemirror_mode": {
-        "name": "ipython",
-        "version": 3
-      },
-      "file_extension": ".py",
-      "mimetype": "text/x-python",
-      "name": "python",
-      "nbconvert_exporter": "python",
-      "pygments_lexer": "ipython3",
-      "version": "3.6.7"
-    },
-    "tags": [
-      "None"
-    ],
-    "task": "Text featurization using DNNs for classification"
-  },
-  "nbformat": 4,
-  "nbformat_minor": 2
-}

how-to-use-azureml/automated-machine-learning/classification-text-dnn/auto-ml-classification-text-dnn.yml

@@ -1,4 +0,0 @@
-name: auto-ml-classification-text-dnn
-dependencies:
-- pip:
-  - azureml-sdk

how-to-use-azureml/automated-machine-learning/classification-text-dnn/helper.py

@@ -1,63 +0,0 @@
-import pandas as pd
-from azureml.core import Environment
-from azureml.core.conda_dependencies import CondaDependencies
-from azureml.train.estimator import Estimator
-from azureml.core.run import Run
-
-
-def run_inference(test_experiment, compute_target, script_folder, train_run,
-                  train_dataset, test_dataset, target_column_name, model_name):
-
-    train_run.download_file('outputs/conda_env_v_1_0_0.yml',
-                            'inference/condafile.yml')
-
-    inference_env = Environment("myenv")
-    inference_env.docker.enabled = True
-    inference_env.python.conda_dependencies = CondaDependencies(
-        conda_dependencies_file_path='inference/condafile.yml')
-
-    est = Estimator(source_directory=script_folder,
-                    entry_script='infer.py',
-                    script_params={
-                        '--target_column_name': target_column_name,
-                        '--model_name': model_name
-                    },
-                    inputs=[
-                        train_dataset.as_named_input('train_data'),
-                        test_dataset.as_named_input('test_data')
-                    ],
-                    compute_target=compute_target,
-                    environment_definition=inference_env)
-
-    run = test_experiment.submit(
-        est, tags={
-            'training_run_id': train_run.id,
-            'run_algorithm': train_run.properties['run_algorithm'],
-            'valid_score': train_run.properties['score'],
-            'primary_metric': train_run.properties['primary_metric']
-        })
-
-    run.log("run_algorithm", run.tags['run_algorithm'])
-    return run
-
-
-def get_result_df(remote_run):
-
-    children = list(remote_run.get_children(recursive=True))
-    summary_df = pd.DataFrame(index=['run_id', 'run_algorithm',
-                                     'primary_metric', 'Score'])
-    goal_minimize = False
-    for run in children:
-        if('run_algorithm' in run.properties and 'score' in run.properties):
-            summary_df[run.id] = [run.id, run.properties['run_algorithm'],
-                                  run.properties['primary_metric'],
-                                  float(run.properties['score'])]
-            if('goal' in run.properties):
-                goal_minimize = run.properties['goal'].split('_')[-1] == 'min'
-
-    summary_df = summary_df.T.sort_values(
-        'Score',
-        ascending=goal_minimize).drop_duplicates(['run_algorithm'])
-    summary_df = summary_df.set_index('run_algorithm')
-
-    return summary_df

how-to-use-azureml/automated-machine-learning/classification-text-dnn/infer.py

@@ -1,60 +0,0 @@
-import argparse
-
-import numpy as np
-
-from sklearn.externals import joblib
-
-from azureml.automl.runtime.shared.score import scoring, constants
-from azureml.core import Run
-from azureml.core.model import Model
-
-
-parser = argparse.ArgumentParser()
-parser.add_argument(
-    '--target_column_name', type=str, dest='target_column_name',
-    help='Target Column Name')
-parser.add_argument(
-    '--model_name', type=str, dest='model_name',
-    help='Name of registered model')
-
-args = parser.parse_args()
-target_column_name = args.target_column_name
-model_name = args.model_name
-
-print('args passed are: ')
-print('Target column name: ', target_column_name)
-print('Name of registered model: ', model_name)
-
-model_path = Model.get_model_path(model_name)
-# deserialize the model file back into a sklearn model
-model = joblib.load(model_path)
-
-run = Run.get_context()
-# get input dataset by name
-test_dataset = run.input_datasets['test_data']
-train_dataset = run.input_datasets['train_data']
-
-X_test_df = test_dataset.drop_columns(columns=[target_column_name]) \
-                        .to_pandas_dataframe()
-y_test_df = test_dataset.with_timestamp_columns(None) \
-                        .keep_columns(columns=[target_column_name]) \
-                        .to_pandas_dataframe()
-y_train_df = test_dataset.with_timestamp_columns(None) \
-                         .keep_columns(columns=[target_column_name]) \
-                         .to_pandas_dataframe()
-
-predicted = model.predict_proba(X_test_df)
-
-# Use the AutoML scoring module
-class_labels = np.unique(np.concatenate((y_train_df.values, y_test_df.values)))
-train_labels = model.classes_
-classification_metrics = list(constants.CLASSIFICATION_SCALAR_SET)
-scores = scoring.score_classification(y_test_df.values, predicted,
-                                      classification_metrics,
-                                      class_labels, train_labels)
-
-print("scores:")
-print(scores)
-
-for key, value in scores.items():
-    run.log(key, value)

how-to-use-azureml/automated-machine-learning/continuous-retraining/auto-ml-continuous-retraining.ipynb

@@ -88,7 +88,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/experimental/README.md

@@ -0,0 +1,92 @@
+# Experimental Notebooks for Automated ML
+Notebooks listed in this folder are leveraging experimental features. Namespaces or function signitures may change in future SDK releases. The notebooks published here will reflect the latest supported APIs. All of these notebooks can run on a client-only installation of the Automated ML SDK.
+The client only installation doesn't contain any of the machine learning libraries, such as scikit-learn, xgboost, or tensorflow, making it much faster to install and is less likely to conflict with any packages in an existing environment. However, since the ML libraries are not available locally, models cannot be downloaded and loaded directly in the client. To replace the functionality of having models locally, these notebooks also demonstrate the ModelProxy feature which will allow you to submit a predict/forecast to the training environment. 
+
+<a name="localconda"></a>
+## Setup using a Local Conda environment
+
+To run these notebook on your own notebook server, use these installation instructions.
+The instructions below will install everything you need and then start a Jupyter notebook.
+If you would like to use a lighter-weight version of the client that does not install all of the machine learning libraries locally, you can leverage the [experimental notebooks.](experimental/README.md)
+
+### 1. Install mini-conda from [here](https://conda.io/miniconda.html), choose 64-bit Python 3.7 or higher.
+- **Note**: if you already have conda installed, you can keep using it but it should be version 4.4.10 or later (as shown by: conda -V).  If you have a previous version installed, you can update it using the command: conda update conda.
+There's no need to install mini-conda specifically.
+
+### 2. Downloading the sample notebooks
+- Download the sample notebooks from [GitHub](https://github.com/Azure/MachineLearningNotebooks) as zip and extract the contents to a local directory.  The automated ML sample notebooks are in the "automated-machine-learning" folder.
+
+### 3. Setup a new conda environment
+The **automl_setup** script creates a new conda environment, installs the necessary packages, configures the widget and starts a jupyter notebook. It takes the conda environment name as an optional parameter.  The default conda environment name is azure_automl.  The exact command depends on the operating system.  See the specific sections below for Windows, Mac and Linux.  It can take about 10 minutes to execute.
+
+Packages installed by the **automl_setup** script:
+    <ul><li>python</li><li>nb_conda</li><li>matplotlib</li><li>numpy</li><li>cython</li><li>urllib3</li><li>pandas</li><li>azureml-sdk</li><li>azureml-widgets</li><li>pandas-ml</li></ul>
+
+For more details refer to the [automl_env.yml](./automl_env.yml)
+## Windows
+Start an **Anaconda Prompt** window, cd to the **how-to-use-azureml/automated-machine-learning/experimental** folder where the sample notebooks were extracted and then run:
+```
+automl_setup
+```
+## Mac
+Install "Command line developer tools" if it is not already installed (you can use the command: `xcode-select --install`).
+
+Start a Terminal windows, cd to the **how-to-use-azureml/automated-machine-learning/experimental** folder where the sample notebooks were extracted and then run:
+
+```
+bash automl_setup_mac.sh
+```
+
+## Linux
+cd to the **how-to-use-azureml/automated-machine-learning/experimental** folder where the sample notebooks were extracted and then run:
+
+```
+bash automl_setup_linux.sh
+```
+
+### 4. Running configuration.ipynb
+- Before running any samples you next need to run the configuration notebook. Click on [configuration](../../configuration.ipynb) notebook
+- Execute the cells in the notebook to Register Machine Learning Services Resource Provider and create a workspace. (*instructions in notebook*)
+
+### 5. Running Samples
+- Please make sure you use the Python [conda env:azure_automl] kernel when trying the sample Notebooks.
+- Follow the instructions in the individual notebooks to explore various features in automated ML.
+
+### 6. Starting jupyter notebook manually
+To start your Jupyter notebook manually, use:
+
+```
+conda activate azure_automl
+jupyter notebook
+```
+
+or on Mac or Linux:
+
+```
+source activate azure_automl
+jupyter notebook
+```
+
+
+<a name="samples"></a>
+# Automated ML SDK Sample Notebooks
+
+- [auto-ml-regression.ipynb](regression/auto-ml-regression.ipynb)
+    - Dataset: Hardware Performance Dataset
+    - Simple example of using automated ML for regression
+    - Uses azure compute for training
+    - Uses ModelProxy for submitting prediction to training environment on azure compute
+
+<a name="documentation"></a>
+See [Configure automated machine learning experiments](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-auto-train) to learn how more about the the settings and features available for automated machine learning experiments.
+
+<a name="pythoncommand"></a>
+# Running using python command
+Jupyter notebook provides a File / Download as / Python (.py) option for saving the notebook as a Python file.
+You can then run this file using the python command.
+However, on Windows the file needs to be modified before it can be run.
+The following condition must be added to the main code in the file:
+
+    if __name__ == "__main__":
+
+The main code of the file must be indented so that it is under this condition.

how-to-use-azureml/automated-machine-learning/experimental/automl_env.yml

@@ -0,0 +1,20 @@
+name: azure_automl_experimental
+dependencies:
+  # The python interpreter version.
+  # Currently Azure ML only supports 3.5.2 and later.
+- pip<=19.3.1
+- python>=3.5.2,<3.8
+- nb_conda
+- matplotlib==2.1.0
+- numpy~=1.18.0
+- cython
+- urllib3<1.24
+- scikit-learn==0.22.1
+- pandas==0.25.1
+
+- pip:
+  # Required packages for AzureML execution, history, and data preparation.
+  - azureml-defaults
+  - azureml-sdk
+  - azureml-widgets
+  - azureml-explain-model

how-to-use-azureml/automated-machine-learning/experimental/automl_env_mac.yml

@@ -0,0 +1,21 @@
+name: azure_automl_experimental
+dependencies:
+  # The python interpreter version.
+  # Currently Azure ML only supports 3.5.2 and later.
+- pip<=19.3.1
+- nomkl
+- python>=3.5.2,<3.8
+- nb_conda
+- matplotlib==2.1.0
+- numpy~=1.18.0
+- cython
+- urllib3<1.24
+- scikit-learn==0.22.1
+- pandas==0.25.1
+
+- pip:
+  # Required packages for AzureML execution, history, and data preparation.
+  - azureml-defaults
+  - azureml-sdk
+  - azureml-widgets
+  - azureml-explain-model

how-to-use-azureml/automated-machine-learning/experimental/automl_setup.cmd

@@ -0,0 +1,63 @@
+@echo off
+set conda_env_name=%1
+set automl_env_file=%2
+set options=%3
+set PIP_NO_WARN_SCRIPT_LOCATION=0
+
+IF "%conda_env_name%"=="" SET conda_env_name="azure_automl_experimental"
+IF "%automl_env_file%"=="" SET automl_env_file="automl_env.yml"
+
+IF NOT EXIST %automl_env_file% GOTO YmlMissing
+
+IF "%CONDA_EXE%"=="" GOTO CondaMissing
+
+call conda activate %conda_env_name% 2>nul:
+
+if not errorlevel 1 (
+  echo Upgrading existing conda environment %conda_env_name%
+  call pip uninstall azureml-train-automl -y -q
+  call conda env update --name %conda_env_name% --file %automl_env_file%
+  if errorlevel 1 goto ErrorExit
+) else (
+  call conda env create -f %automl_env_file% -n %conda_env_name%
+)
+
+call conda activate %conda_env_name% 2>nul:
+if errorlevel 1 goto ErrorExit
+
+call python -m ipykernel install --user --name %conda_env_name% --display-name "Python (%conda_env_name%)"
+
+REM azureml.widgets is now installed as part of the pip install under the conda env.
+REM Removing the old user install so that the notebooks will use the latest widget.
+call jupyter nbextension uninstall --user --py azureml.widgets
+
+echo.
+echo.
+echo ***************************************
+echo * AutoML setup completed successfully *
+echo ***************************************
+IF NOT "%options%"=="nolaunch" (
+  echo.
+  echo Starting jupyter notebook - please run the configuration notebook 
+  echo.
+  jupyter notebook --log-level=50 --notebook-dir='..\..'
+)
+
+goto End
+
+:CondaMissing
+echo Please run this script from an Anaconda Prompt window.
+echo You can start an Anaconda Prompt window by
+echo typing Anaconda Prompt on the Start menu.
+echo If you don't see the Anaconda Prompt app, install Miniconda.
+echo If you are running an older version of Miniconda or Anaconda,
+echo you can upgrade using the command: conda update conda
+goto End
+
+:YmlMissing
+echo File %automl_env_file% not found.
+
+:ErrorExit
+echo Install failed
+
+:End

how-to-use-azureml/automated-machine-learning/experimental/automl_setup_linux.sh

@@ -0,0 +1,53 @@
+#!/bin/bash
+
+CONDA_ENV_NAME=$1
+AUTOML_ENV_FILE=$2
+OPTIONS=$3
+PIP_NO_WARN_SCRIPT_LOCATION=0
+
+if [ "$CONDA_ENV_NAME" == "" ]
+then
+  CONDA_ENV_NAME="azure_automl_experimental"
+fi
+
+if [ "$AUTOML_ENV_FILE" == "" ]
+then
+  AUTOML_ENV_FILE="automl_env.yml"
+fi
+
+if [ ! -f $AUTOML_ENV_FILE ]; then
+    echo "File $AUTOML_ENV_FILE not found"
+    exit 1
+fi
+
+if source activate $CONDA_ENV_NAME 2> /dev/null
+then
+   echo "Upgrading existing conda environment" $CONDA_ENV_NAME
+   pip uninstall azureml-train-automl -y -q
+   conda env update --name $CONDA_ENV_NAME --file $AUTOML_ENV_FILE &&
+   jupyter nbextension uninstall --user --py azureml.widgets
+else
+   conda env create -f $AUTOML_ENV_FILE -n $CONDA_ENV_NAME &&
+   source activate $CONDA_ENV_NAME &&
+   python -m ipykernel install --user --name $CONDA_ENV_NAME --display-name "Python ($CONDA_ENV_NAME)" &&
+   jupyter nbextension uninstall --user --py azureml.widgets &&
+   echo "" &&
+   echo "" &&
+   echo "***************************************" &&
+   echo "* AutoML setup completed successfully *" &&
+   echo "***************************************" &&
+   if [ "$OPTIONS" != "nolaunch" ]
+   then
+      echo "" &&
+      echo "Starting jupyter notebook - please run the configuration notebook" &&
+      echo "" &&
+      jupyter notebook --log-level=50 --notebook-dir '../..'
+   fi
+fi
+
+if [ $? -gt 0 ]
+then
+   echo "Installation failed"
+fi
+
+

how-to-use-azureml/automated-machine-learning/experimental/automl_setup_mac.sh

@@ -0,0 +1,55 @@
+#!/bin/bash
+
+CONDA_ENV_NAME=$1
+AUTOML_ENV_FILE=$2
+OPTIONS=$3
+PIP_NO_WARN_SCRIPT_LOCATION=0
+
+if [ "$CONDA_ENV_NAME" == "" ]
+then
+  CONDA_ENV_NAME="azure_automl_experimental"
+fi
+
+if [ "$AUTOML_ENV_FILE" == "" ]
+then
+  AUTOML_ENV_FILE="automl_env.yml"
+fi
+
+if [ ! -f $AUTOML_ENV_FILE ]; then
+    echo "File $AUTOML_ENV_FILE not found"
+    exit 1
+fi
+
+if source activate $CONDA_ENV_NAME 2> /dev/null
+then
+   echo "Upgrading existing conda environment" $CONDA_ENV_NAME
+   pip uninstall azureml-train-automl -y -q
+   conda env update --name $CONDA_ENV_NAME --file $AUTOML_ENV_FILE &&
+   jupyter nbextension uninstall --user --py azureml.widgets
+else
+   conda env create -f $AUTOML_ENV_FILE -n $CONDA_ENV_NAME &&
+   source activate $CONDA_ENV_NAME &&
+   conda install lightgbm -c conda-forge -y &&
+   python -m ipykernel install --user --name $CONDA_ENV_NAME --display-name "Python ($CONDA_ENV_NAME)" &&
+   jupyter nbextension uninstall --user --py azureml.widgets &&
+   echo "" &&
+   echo "" &&
+   echo "***************************************" &&
+   echo "* AutoML setup completed successfully *" &&
+   echo "***************************************" &&
+   if [ "$OPTIONS" != "nolaunch" ]
+   then
+      echo "" &&
+      echo "Starting jupyter notebook - please run the configuration notebook" &&
+      echo "" &&
+      jupyter notebook --log-level=50 --notebook-dir '../..'
+   fi
+fi
+
+if [ $? -gt 0 ]
+then
+   echo "Installation failed"
+fi
+
+
+

how-to-use-azureml/automated-machine-learning/experimental/regression/auto-ml-regression-model-proxy.ipynb

@@ -0,0 +1,481 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
+        "\n",
+        "Licensed under the MIT License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/automated-machine-learning/experimental/regression/auto-ml-regression.png)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# Automated Machine Learning\n",
+        "_**Regression with Aml Compute**_\n",
+        "\n",
+        "## Contents\n",
+        "1. [Introduction](#Introduction)\n",
+        "1. [Setup](#Setup)\n",
+        "1. [Data](#Data)\n",
+        "1. [Train](#Train)\n",
+        "1. [Results](#Results)\n",
+        "1. [Test](#Test)\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Introduction\n",
+        "In this example we use the Hardware Performance Dataset to showcase how you can use AutoML for a simple regression problem. The Regression goal is to predict the performance of certain combinations of hardware parts.\n",
+        "\n",
+        "If you are using an Azure Machine Learning Compute Instance, you are all set.  Otherwise, go through the [configuration](../../../../configuration.ipynb)  notebook first if you haven't already to establish your connection to the AzureML Workspace. \n",
+        "\n",
+        "In this notebook you will learn how to:\n",
+        "1. Create an `Experiment` in an existing `Workspace`.\n",
+        "2. Configure AutoML using `AutoMLConfig`.\n",
+        "3. Train the model using remote compute.\n",
+        "4. Explore the results.\n",
+        "5. Test the best fitted model."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Setup\n",
+        "\n",
+        "As part of the setup you have already created an Azure ML `Workspace` object. For Automated ML you will need to create an `Experiment` object, which is a named object in a `Workspace` used to run experiments."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "import logging\n",
+        "\n",
+        "from matplotlib import pyplot as plt\n",
+        "import numpy as np\n",
+        "import pandas as pd\n",
+        " \n",
+        "\n",
+        "import azureml.core\n",
+        "from azureml.core.experiment import Experiment\n",
+        "from azureml.core.workspace import Workspace\n",
+        "from azureml.core.dataset import Dataset\n",
+        "from azureml.train.automl import AutoMLConfig"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "This sample notebook may use features that are not available in previous versions of the Azure ML SDK."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
+        "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "ws = Workspace.from_config()\n",
+        "\n",
+        "# Choose a name for the experiment.\n",
+        "experiment_name = 'automl-regression-model-proxy'\n",
+        "\n",
+        "experiment = Experiment(ws, experiment_name)\n",
+        "\n",
+        "output = {}\n",
+        "output['Subscription ID'] = ws.subscription_id\n",
+        "output['Workspace'] = ws.name\n",
+        "output['Resource Group'] = ws.resource_group\n",
+        "output['Location'] = ws.location\n",
+        "output['Run History Name'] = experiment_name\n",
+        "pd.set_option('display.max_colwidth', -1)\n",
+        "outputDf = pd.DataFrame(data = output, index = [''])\n",
+        "outputDf.T"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Using AmlCompute\n",
+        "You will need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture#compute-target) for your AutoML run. In this tutorial, you use `AmlCompute` as your training compute resource."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from azureml.core.compute import ComputeTarget, AmlCompute\n",
+        "from azureml.core.compute_target import ComputeTargetException\n",
+        "\n",
+        "# Choose a name for your CPU cluster\n",
+        "cpu_cluster_name = \"reg-cluster\"\n",
+        "\n",
+        "# Verify that cluster does not exist already\n",
+        "try:\n",
+        "    compute_target = ComputeTarget(workspace=ws, name=cpu_cluster_name)\n",
+        "    print('Found existing cluster, use it.')\n",
+        "except ComputeTargetException:\n",
+        "    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',\n",
+        "                                                           max_nodes=4)\n",
+        "    compute_target = ComputeTarget.create(ws, cpu_cluster_name, compute_config)\n",
+        "\n",
+        "compute_target.wait_for_completion(show_output=True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Data\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Load Data\n",
+        "Load the hardware dataset from a csv file containing both training features and labels. The features are inputs to the model, while the training labels represent the expected output of the model. Next, we'll split the data using random_split and extract the training data for the model. "
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "data = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/machineData.csv\"\n",
+        "dataset = Dataset.Tabular.from_delimited_files(data)\n",
+        "\n",
+        "# Split the dataset into train and test datasets\n",
+        "train_data, test_data = dataset.random_split(percentage=0.8, seed=223)\n",
+        "\n",
+        "label = \"ERP\"\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Train\n",
+        "\n",
+        "Instantiate an `AutoMLConfig` object to specify the settings and data used to run the experiment.\n",
+        "\n",
+        "|Property|Description|\n",
+        "|-|-|\n",
+        "|**task**|classification, regression or forecasting|\n",
+        "|**primary_metric**|This is the metric that you want to optimize. Regression supports the following primary metrics: <br><i>spearman_correlation</i><br><i>normalized_root_mean_squared_error</i><br><i>r2_score</i><br><i>normalized_mean_absolute_error</i>|\n",
+        "|**n_cross_validations**|Number of cross validation splits.|\n",
+        "|**training_data**|(sparse) array-like, shape = [n_samples, n_features]|\n",
+        "|**label_column_name**|(sparse) array-like, shape = [n_samples, ], targets values.|\n",
+        "\n",
+        "**_You can find more information about primary metrics_** [here](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-auto-train#primary-metric)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": [
+          "automlconfig-remarks-sample"
+        ]
+      },
+      "outputs": [],
+      "source": [
+        "automl_settings = {\n",
+        "    \"n_cross_validations\": 3,\n",
+        "    \"primary_metric\": 'r2_score',\n",
+        "    \"enable_early_stopping\": True, \n",
+        "    \"experiment_timeout_hours\": 0.3, #for real scenarios we reccommend a timeout of at least one hour \n",
+        "    \"max_concurrent_iterations\": 4,\n",
+        "    \"max_cores_per_iteration\": -1,\n",
+        "    \"verbosity\": logging.INFO,\n",
+        "}\n",
+        "\n",
+        "automl_config = AutoMLConfig(task = 'regression',\n",
+        "                             compute_target = compute_target,\n",
+        "                             training_data = train_data,\n",
+        "                             label_column_name = label,\n",
+        "                             **automl_settings\n",
+        "                            )"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Call the `submit` method on the experiment object and pass the run configuration. Execution of remote runs is asynchronous. Depending on the data and the number of iterations this can run for a while.  Validation errors and current status will be shown when setting `show_output=True` and the execution will be synchronous."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "remote_run = experiment.submit(automl_config, show_output = False)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# If you need to retrieve a run that already started, use the following code\n",
+        "#from azureml.train.automl.run import AutoMLRun\n",
+        "#remote_run = AutoMLRun(experiment = experiment, run_id = '<replace with your run id>')"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "remote_run"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Results"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "#### Widget for Monitoring Runs\n",
+        "\n",
+        "The widget will first report a \"loading\" status while running the first iteration. After completing the first iteration, an auto-updating graph and table will be shown. The widget will refresh once per minute, so you should see the graph update as child runs complete.\n",
+        "\n",
+        "**Note:** The widget displays a link at the bottom. Use this link to open a web interface to explore the individual run details."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from azureml.widgets import RunDetails\n",
+        "RunDetails(remote_run).show() "
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "remote_run.wait_for_completion()"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Retrieve the Best Child Run\n",
+        "\n",
+        "Below we select the best pipeline from our iterations. The `get_best_child` method returns the best run. Overloads on `get_best_child` allow you to retrieve the best run for *any* logged metric."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "best_run = remote_run.get_best_child()\n",
+        "print(best_run)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "#### Best Child Run Based on Any Other Metric\n",
+        "Show the run and the model that has the smallest `root_mean_squared_error` value (which turned out to be the same as the one with largest `spearman_correlation` value):"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "lookup_metric = \"root_mean_squared_error\"\n",
+        "best_run = remote_run.get_best_child(metric = lookup_metric)\n",
+        "print(best_run)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# preview the first 3 rows of the dataset\n",
+        "\n",
+        "test_data = test_data.to_pandas_dataframe()\n",
+        "y_test = test_data['ERP'].fillna(0)\n",
+        "test_data = test_data.drop('ERP', 1)\n",
+        "test_data = test_data.fillna(0)\n",
+        "\n",
+        "\n",
+        "train_data = train_data.to_pandas_dataframe()\n",
+        "y_train = train_data['ERP'].fillna(0)\n",
+        "train_data = train_data.drop('ERP', 1)\n",
+        "train_data = train_data.fillna(0)\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "#### Creating ModelProxy for submitting prediction runs to the training environment.\n",
+        "We will create a ModelProxy for the best child run, which will allow us to submit a run that does the prediction in the training environment. Unlike the local client, which can have different versions of some libraries, the training environment will have all the compatible libraries for the model already."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from azureml.train.automl.model_proxy import ModelProxy\n",
+        "best_model_proxy = ModelProxy(best_run)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "y_pred_train = best_model_proxy.predict(train_data).to_pandas_dataframe()\n",
+        "y_residual_train = y_train - y_pred_train\n",
+        "\n",
+        "y_pred_test = best_model_proxy.predict(test_data).to_pandas_dataframe()\n",
+        "y_residual_test = y_test - y_pred_test"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "%matplotlib inline\n",
+        "from sklearn.metrics import mean_squared_error, r2_score\n",
+        "\n",
+        "# Set up a multi-plot chart.\n",
+        "f, (a0, a1) = plt.subplots(1, 2, gridspec_kw = {'width_ratios':[1, 1], 'wspace':0, 'hspace': 0})\n",
+        "f.suptitle('Regression Residual Values', fontsize = 18)\n",
+        "f.set_figheight(6)\n",
+        "f.set_figwidth(16)\n",
+        "\n",
+        "# Plot residual values of training set.\n",
+        "a0.axis([0, 360, -100, 100])\n",
+        "a0.plot(y_residual_train, 'bo', alpha = 0.5)\n",
+        "a0.plot([-10,360],[0,0], 'r-', lw = 3)\n",
+        "a0.text(16,170,'RMSE = {0:.2f}'.format(np.sqrt(mean_squared_error(y_train, y_pred_train))), fontsize = 12)\n",
+        "a0.text(16,140,'R2 score = {0:.2f}'.format(r2_score(y_train, y_pred_train)),fontsize = 12)\n",
+        "a0.set_xlabel('Training samples', fontsize = 12)\n",
+        "a0.set_ylabel('Residual Values', fontsize = 12)\n",
+        "\n",
+        "# Plot residual values of test set.\n",
+        "a1.axis([0, 90, -100, 100])\n",
+        "a1.plot(y_residual_test, 'bo', alpha = 0.5)\n",
+        "a1.plot([-10,360],[0,0], 'r-', lw = 3)\n",
+        "a1.text(5,170,'RMSE = {0:.2f}'.format(np.sqrt(mean_squared_error(y_test, y_pred_test))), fontsize = 12)\n",
+        "a1.text(5,140,'R2 score = {0:.2f}'.format(r2_score(y_test, y_pred_test)),fontsize = 12)\n",
+        "a1.set_xlabel('Test samples', fontsize = 12)\n",
+        "a1.set_yticklabels([])\n",
+        "\n",
+        "plt.show()"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "%matplotlib inline\n",
+        "test_pred = plt.scatter(y_test, y_pred_test, color='')\n",
+        "test_test = plt.scatter(y_test, y_test, color='g')\n",
+        "plt.legend((test_pred, test_test), ('prediction', 'truth'), loc='upper left', fontsize=8)\n",
+        "plt.show()"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": []
+    }
+  ],
+  "metadata": {
+    "authors": [
+      {
+        "name": "rakellam"
+      }
+    ],
+    "categories": [
+      "how-to-use-azureml",
+      "automated-machine-learning"
+    ],
+    "kernelspec": {
+      "display_name": "Python 3.6",
+      "language": "python",
+      "name": "python36"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.6.2"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 2
+}

how-to-use-azureml/automated-machine-learning/experimental/regression/auto-ml-regression-model-proxy.yml

@@ -0,0 +1,4 @@
+name: auto-ml-regression-model-proxy
+dependencies:
+- pip:
+  - azureml-sdk

how-to-use-azureml/automated-machine-learning/forecasting-beer-remote/auto-ml-forecasting-beer-remote.ipynb

@@ -114,7 +114,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-bike-share/auto-ml-forecasting-bike-share.ipynb

@@ -87,7 +87,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-energy-demand/auto-ml-forecasting-energy-demand.ipynb

@@ -97,7 +97,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-forecast-function/auto-ml-forecasting-function.ipynb

@@ -94,7 +94,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-orange-juice-sales/auto-ml-forecasting-orange-juice-sales.ipynb

@@ -82,7 +82,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -327,7 +327,7 @@
         "\n",
         "The featurization customization in forecasting is an advanced feature in AutoML which allows our customers to change the default forecasting featurization behaviors and column types through `FeaturizationConfig`. The supported scenarios include,\n",
         "1. Column purposes update: Override feature type for the specified column. Currently supports DateTime, Categorical and Numeric. This customization can be used in the scenario that the type of the column cannot correctly reflect its purpose. Some numerical columns, for instance, can be treated as Categorical columns which need to be converted to categorical while some can be treated as epoch timestamp which need to be converted to datetime. To tell our SDK to correctly preprocess these columns, a configuration need to be add with the columns and their desired types.\n",
-        "2. Transformer parameters update: Currently supports parameter change for Imputer only. User can customize imputation methods, the supported methods are constant for target data and mean, median, most frequent and constant for training data. This customization can be used for the scenario that our customers know which imputation methods fit best to the input data. For instance, some datasets use NaN to represent 0 which the correct behavior should impute all the missing value with 0. To achieve this behavior, these columns need to be configured as constant imputation with `fill_value` 0.\n",
+        "2. Transformer parameters update: Currently supports parameter change for Imputer only. User can customize imputation methods. The supported imputing methods for target column are constant and ffill (forward fill). The supported imputing methods for feature columns are mean, median, most frequent, constant and ffill (forward fill). This customization can be used for the scenario that our customers know which imputation methods fit best to the input data. For instance, some datasets use NaN to represent 0 which the correct behavior should impute all the missing value with 0. To achieve this behavior, these columns need to be configured as constant imputation with `fill_value` 0.\n",
         "3. Drop columns: Columns to drop from being featurized. These usually are the columns which are leaky or the columns contain no useful data.\n",
         "\n",
         "This step requires an Enterprise workspace to gain access to this feature. To learn more about creating an Enterprise workspace or upgrading to an Enterprise workspace from the Azure portal, please visit our [Workspace page.](https://docs.microsoft.com/azure/machine-learning/service/concept-workspace#upgrade)"
@@ -350,7 +350,9 @@
         "# Fill missing values in the target column, Quantity, with zeros.\n",
         "featurization_config.add_transformer_params('Imputer', ['Quantity'], {\"strategy\": \"constant\", \"fill_value\": 0})\n",
         "# Fill missing values in the INCOME column with median value.\n",
-        "featurization_config.add_transformer_params('Imputer', ['INCOME'], {\"strategy\": \"median\"})"
+        "featurization_config.add_transformer_params('Imputer', ['INCOME'], {\"strategy\": \"median\"})\n",
+        "# Fill missing values in the Price column with forward fill (last value carried forward).\n",
+        "featurization_config.add_transformer_params('Imputer', ['Price'], {\"strategy\": \"ffill\"})"
       ]
     },
     {

how-to-use-azureml/automated-machine-learning/local-run-classification-credit-card-fraud/auto-ml-classification-credit-card-fraud-local.ipynb

@@ -96,7 +96,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -562,16 +562,10 @@
       "outputs": [],
       "source": [
         "%%writefile score.py\n",
-        "import numpy as np\n",
-        "import pandas as pd\n",
-        "import os\n",
-        "import pickle\n",
-        "import azureml.train.automl\n",
-        "import azureml.interpret\n",
-        "from azureml.train.automl.runtime.automl_explain_utilities import AutoMLExplainerSetupClass, \\\n",
-        "    automl_setup_model_explanations\n",
         "import joblib\n",
+        "import pandas as pd\n",
         "from azureml.core.model import Model\n",
+        "from azureml.train.automl.runtime.automl_explain_utilities import automl_setup_model_explanations\n",
         "\n",
         "\n",
         "def init():\n",

how-to-use-azureml/automated-machine-learning/regression-explanation-featurization/auto-ml-regression-explanation-featurization.ipynb

@@ -98,7 +98,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/regression-explanation-featurization/score_explain.py

@@ -1,14 +1,7 @@
-import json
-import numpy as np
 import pandas as pd
-import os
-import pickle
-import azureml.train.automl
-import azureml.interpret
-from azureml.train.automl.runtime.automl_explain_utilities import AutoMLExplainerSetupClass, \
-    automl_setup_model_explanations
 import joblib
 from azureml.core.model import Model
+from azureml.train.automl.runtime.automl_explain_utilities import automl_setup_model_explanations
 
 
 def init():

how-to-use-azureml/automated-machine-learning/regression-explanation-featurization/train_explainer.py

@@ -1,17 +1,17 @@
 # Copyright (c) Microsoft. All rights reserved.
 # Licensed under the MIT license.
 import os
+import joblib
 
-from azureml.core.run import Run
+from interpret.ext.glassbox import LGBMExplainableModel
+from automl.client.core.common.constants import MODEL_PATH
 from azureml.core.experiment import Experiment
 from azureml.core.dataset import Dataset
-from azureml.train.automl.runtime.automl_explain_utilities import AutoMLExplainerSetupClass, \
-    automl_setup_model_explanations, automl_check_model_if_explainable
-from interpret.ext.glassbox import LGBMExplainableModel
+from azureml.core.run import Run
 from azureml.interpret.mimic_wrapper import MimicWrapper
-from automl.client.core.common.constants import MODEL_PATH
 from azureml.interpret.scoring.scoring_explainer import TreeScoringExplainer
-import joblib
+from azureml.train.automl.runtime.automl_explain_utilities import automl_setup_model_explanations, \
+    automl_check_model_if_explainable
 
 
 OUTPUT_DIR = './outputs/'

how-to-use-azureml/automated-machine-learning/regression/auto-ml-regression.ipynb

@@ -92,7 +92,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.12.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.14.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/azure-databricks/automl/README.md

@@ -0,0 +1,56 @@
+# Adding an init script to an Azure Databricks cluster
+
+The [azureml-cluster-init.sh](./azureml-cluster-init.sh) script configures the environment to
+1. Install the latest AutoML library
+
+To create the Azure Databricks cluster-scoped init script
+
+1. Create the base directory you want to store the init script in if it does not exist.
+    ```
+    dbutils.fs.mkdirs("dbfs:/databricks/init/")
+    ```
+
+2. Create the script azureml-cluster-init.sh
+    ```
+    dbutils.fs.put("/databricks/init/azureml-cluster-init.sh","""
+    #!/bin/bash
+	set -ex
+	/databricks/python/bin/pip install -r https://aka.ms/automl_linux_requirements.txt
+    """, True)
+    ```
+
+3. Check that the script exists.
+    ```
+    display(dbutils.fs.ls("dbfs:/databricks/init/azureml-cluster-init.sh"))
+    ```
+
+1. Configure the cluster to run the script.
+    * Using the cluster configuration page
+        1. On the cluster configuration page, click the Advanced Options toggle.
+        1. At the bottom of the page, click the Init Scripts tab.
+        1. In the Destination drop-down, select a destination type. Example: 'DBFS'
+        1. Specify a path to the init script.
+            ```
+            dbfs:/databricks/init/azureml-cluster-init.sh
+            ```
+        1. Click Add
+
+    * Using the API.
+        ```
+        curl -n -X POST -H 'Content-Type: application/json' -d '{
+        "cluster_id": "<cluster_id>",
+        "num_workers": <num_workers>,
+        "spark_version": "<spark_version>",
+        "node_type_id": "<node_type_id>",
+        "cluster_log_conf": {
+            "dbfs" : {
+            "destination": "dbfs:/cluster-logs"
+            }
+        },
+        "init_scripts": [ {
+            "dbfs": {
+            "destination": "dbfs:/databricks/init/azureml-cluster-init.sh"
+            }
+        } ]
+        }' https://<databricks-instance>/api/2.0/clusters/edit
+        ```

how-to-use-azureml/azure-databricks/automl/automl-databricks-local-01.ipynb

@@ -13,12 +13,13 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "We support installing AML SDK as library from GUI. When attaching a library follow this https://docs.databricks.com/user-guide/libraries.html and add the below string as your PyPi package. You can select the option to attach the library to all clusters or just one cluster.\n",
+        "## AutoML Installation\n",
         "\n",
-        "**install azureml-sdk with Automated ML**\n",
-        "* Source: Upload Python Egg or PyPi\n",
-        "* PyPi Name: `azureml-sdk[automl]`\n",
-        "* Select Install Library"
+        "**For Databricks non ML runtime 7.1(scala 2.21, spark 3.0.0) and up, Install AML sdk by running the following command in the first cell of the notebook.**\n",
+        "\n",
+        "%pip install -r https://aka.ms/automl_linux_requirements.txt\n",
+        "\n",
+        "**For Databricks non ML runtime 7.0 and lower, Install AML sdk using init script as shown in [readme](readme.md) before running this notebook.**\n"
       ]
     },
     {

how-to-use-azureml/azure-databricks/automl/automl-databricks-local-with-deployment.ipynb

@@ -13,12 +13,13 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "We support installing AML SDK as library from GUI. When attaching a library follow this https://docs.databricks.com/user-guide/libraries.html and add the below string as your PyPi package. You can select the option to attach the library to all clusters or just one cluster.\n",
+        "## AutoML Installation\n",
         "\n",
-        "**install azureml-sdk with Automated ML**\n",
-        "* Source: Upload Python Egg or PyPi\n",
-        "* PyPi Name: `azureml-sdk[automl]`\n",
-        "* Select Install Library"
+        "**For Databricks non ML runtime 7.1(scala 2.21, spark 3.0.0) and up, Install AML sdk by running the following command in the first cell of the notebook.**\n",
+        "\n",
+        "%pip install -r https://aka.ms/automl_linux_requirements.txt\n",
+        "\n",
+        "**For Databricks non ML runtime 7.0 and lower, Install AML sdk using init script as shown in [readme](readme.md) before running this notebook.**"
       ]
     },
     {

how-to-use-azureml/azureml-sdk-for-r/README.md

@@ -1,36 +0,0 @@
-## Examples to get started with Azure Machine Learning SDK for R
-
-Learn how to use Azure Machine Learning SDK for R for experimentation and model management.
-
-As a pre-requisite, go through the [Installation](vignettes/installation.Rmd) and [Configuration](vignettes/configuration.Rmd) vignettes to first install the package and set up your Azure Machine Learning Workspace unless you are running these examples on an Azure Machine Learning compute instance. Azure Machine Learning compute instances have the Azure Machine Learning SDK pre-installed and your workspace details pre-configured.
-
-
-Samples
-* Deployment
-	* [deploy-to-aci](./samples/deployment/deploy-to-aci): Deploy a model as a web service to Azure Container Instances (ACI).
-	* [deploy-to-local](./samples/deployment/deploy-to-local): Deploy a model as a web service locally.
-* Training
-	* [train-on-amlcompute](./samples/training/train-on-amlcompute): Train a model on a remote AmlCompute cluster.
-	* [train-on-local](./samples/training/train-on-local): Train a model locally with Docker.
-
-Vignettes
-* [deploy-to-aks](./vignettes/deploy-to-aks): Production deploy a model as a web service to Azure Kubernetes Service (AKS).
-* [hyperparameter-tune-with-keras](./vignettes/hyperparameter-tune-with-keras): Hyperparameter tune a Keras model using HyperDrive, Azure ML's hyperparameter tuning functionality.
-* [train-and-deploy-to-aci](./vignettes/train-and-deploy-to-aci): Train a caret model and deploy as a web service to Azure Container Instances (ACI).
-* [train-with-tensorflow](./vignettes/train-with-tensorflow): Train a deep learning TensorFlow model with Azure ML.
-
-Find more information on the [official documentation site for Azure Machine Learning SDK for R](https://azure.github.io/azureml-sdk-for-r/).
-
-
-### Troubleshooting
-
-- If the following error occurs when submitting an experiment using RStudio:
-   ```R
-    Error in py_call_impl(callable, dots$args, dots$keywords) : 
-     PermissionError: [Errno 13] Permission denied
-   ```
-  Move the files for your project into a subdirectory and reset the working directory to that directory before re-submitting.
-  
-  In order to submit an experiment, the Azure ML SDK must create a .zip file of the project directory to send to the service. However,
-  the SDK does not have permission to write into the .Rproj.user subdirectory that is automatically created during an RStudio
-  session. For this reason, the recommended best practice is to isolate project files into their own directory.

how-to-use-azureml/azureml-sdk-for-r/samples/README.md

@@ -1,11 +0,0 @@
-## Azure Machine Learning samples
-These samples are short code examples for using Azure Machine Learning SDK for R. If you are new to the R SDK, we recommend that you first take a look at the more detailed end-to-end [vignettes](../vignettes).
-
-Before running a sample in RStudio, set the working directory to the folder that contains the sample script in RStudio using `setwd(dirname)` or Session -> Set Working Directory -> To Source File Location. Each vignette assumes that the data and scripts are in the current working directory.
-
-1. [train-on-amlcompute](training/train-on-amlcompute): Train a model on a remote AmlCompute cluster.
-2. [train-on-local](training/train-on-local): Train a model locally with Docker.
-2. [deploy-to-aci](deployment/deploy-to-aci): Deploy a model as a web service to Azure Container Instances (ACI).
-3. [deploy-to-local](deployment/deploy-to-local): Deploy a model as a web service locally.
-
-> Before you run these samples, make sure you have an Azure Machine Learning workspace. You can follow the [configuration vignette](../vignettes/configuration.Rmd) to set up a workspace. (You do not need to do this if you are running these examples on an Azure Machine Learning compute instance).

how-to-use-azureml/azureml-sdk-for-r/samples/deployment/deploy-to-aci/deploy-to-aci.R

@@ -1,59 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-library(azuremlsdk)
-library(jsonlite)
-
-ws <- load_workspace_from_config()
-
-# Register the model
-model <- register_model(ws, model_path = "project_files/model.rds",
-                        model_name = "model.rds")
-
-# Create environment
-r_env <- r_environment(name = "r_env")
-
-# Create inference config
-inference_config <- inference_config(
-  entry_script = "score.R",
-  source_directory = "project_files",
-  environment = r_env)
-
-# Create ACI deployment config
-deployment_config <- aci_webservice_deployment_config(cpu_cores = 1,
-                                                      memory_gb = 1)
-
-# Deploy the web service
-service <- deploy_model(ws,
-                        'rservice',
-                        list(model),
-                        inference_config,
-                        deployment_config)
-wait_for_deployment(service, show_output = TRUE)
-
-# If you encounter any issue in deploying the webservice, please visit
-# https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-troubleshoot-deployment
-
-# Inferencing
-# versicolor
-plant <- data.frame(Sepal.Length = 6.4,
-                    Sepal.Width = 2.8,
-                    Petal.Length = 4.6,
-                    Petal.Width = 1.8)
-# setosa
-plant <- data.frame(Sepal.Length = 5.1,
-                    Sepal.Width = 3.5,
-                    Petal.Length = 1.4,
-                    Petal.Width = 0.2)
-# virginica
-plant <- data.frame(Sepal.Length = 6.7,
-                    Sepal.Width = 3.3,
-                    Petal.Length = 5.2,
-                    Petal.Width = 2.3)
-
-# Test the web service
-predicted_val <- invoke_webservice(service, toJSON(plant))
-predicted_val
-
-# Delete the web service
-delete_webservice(service)

how-to-use-azureml/azureml-sdk-for-r/samples/deployment/deploy-to-aci/project_files/score.R

@@ -1,17 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-library(jsonlite)
-
-init <- function() {
-    model_path <- Sys.getenv("AZUREML_MODEL_DIR")
-    model <- readRDS(file.path(model_path, "model.rds"))
-    message("model is loaded")
-
-    function(data) {
-        plant <- as.data.frame(fromJSON(data))
-        prediction <- predict(model, plant)
-        result <- as.character(prediction)
-        toJSON(result)
-    }
-}

how-to-use-azureml/azureml-sdk-for-r/samples/deployment/deploy-to-local/deploy-to-local.R

@@ -1,112 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-# Register model and deploy locally
-# This example shows how to deploy a web service in step-by-step fashion:
-#   
-# 1) Register model
-# 2) Deploy the model as a web service in a local Docker container.
-# 3) Invoke web service with SDK or call web service with raw HTTP call.
-# 4) Quickly test changes to your entry script by reloading the local service.
-# 5) Optionally, you can also make changes to model and update the local service.
-
-library(azuremlsdk)
-library(jsonlite)
-
-ws <- load_workspace_from_config()
-
-# Register the model
-model <- register_model(ws, model_path = "project_files/model.rds",
-                        model_name = "model.rds")
-
-# Create environment
-r_env <- r_environment(name = "r_env")
-
-# Create inference config
-inference_config <- inference_config(
-  entry_script = "score.R",
-  source_directory = "project_files",
-  environment = r_env)
-
-# Create local deployment config
-local_deployment_config <- local_webservice_deployment_config()
-
-# Deploy the web service
-# NOTE:
-# The Docker image runs as a Linux container. If you are running Docker for Windows, you need to ensure the Linux Engine is running:
-# # PowerShell command to switch to Linux engine
-# & 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchLinuxEngine
-service <- deploy_model(ws,
-                        'rservice-local',
-                        list(model),
-                        inference_config,
-                        local_deployment_config)
-# Wait for deployment
-wait_for_deployment(service, show_output = TRUE)
-
-# Show the port of local service
-message(service$port)
-
-# If you encounter any issue in deploying the webservice, please visit
-# https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-troubleshoot-deployment
-
-# Inferencing
-# versicolor
-# plant <- data.frame(Sepal.Length = 6.4,
-#                     Sepal.Width = 2.8,
-#                     Petal.Length = 4.6,
-#                     Petal.Width = 1.8)
-# setosa
-plant <- data.frame(Sepal.Length = 5.1,
-                    Sepal.Width = 3.5,
-                    Petal.Length = 1.4,
-                    Petal.Width = 0.2)
-# # virginica
-# plant <- data.frame(Sepal.Length = 6.7,
-#                     Sepal.Width = 3.3,
-#                     Petal.Length = 5.2,
-#                     Petal.Width = 2.3)
-
-#Test the web service
-invoke_webservice(service, toJSON(plant))
-
-## The last few lines of the logs should have the correct prediction and should display -> R[write to console]: "setosa" 
-cat(gsub(pattern = "\n", replacement = " \n", x = get_webservice_logs(service)))
-
-## Test the web service with a HTTP Raw request
-# 
-# NOTE:
-# To test the service locally use the https://localhost:<local_service$port> URL
-
-# Import the request library
-library(httr)
-# Get the service scoring URL from the service object, its URL is for testing locally
-local_service_url <- service$scoring_uri #Same as https://localhost:<local_service$port>
-
-#POST request to web service
-resp <- POST(local_service_url, body = plant, encode = "json", verbose())
-
-## The last few lines of the logs should have the correct prediction and should display -> R[write to console]: "setosa" 
-cat(gsub(pattern = "\n", replacement = " \n", x = get_webservice_logs(service)))
-
-
-# Optional, use a new scoring script
-inference_config <- inference_config(
-  entry_script = "score_new.R",
-  source_directory = "project_files",
-  environment = r_env)
-
-## Then reload the service to see the changes made
-reload_local_webservice_assets(service)
-
-## Check reloaded service, you will see the last line will say "this is a new scoring script! I was reloaded"
-invoke_webservice(service, toJSON(plant))
-cat(gsub(pattern = "\n", replacement = " \n", x = get_webservice_logs(service)))
-
-# Update service
-# If you want to change your model(s), environment, or deployment configuration, call update() to rebuild the Docker image.
-
-# update_local_webservice(service, models = [NewModelObject], deployment_config = deployment_config, wait = FALSE, inference_config = inference_config)
-
-# Delete service
-delete_local_webservice(service)

how-to-use-azureml/azureml-sdk-for-r/samples/deployment/deploy-to-local/project_files/score.R

@@ -1,18 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-library(jsonlite)
-
-init <- function() {
-    model_path <- Sys.getenv("AZUREML_MODEL_DIR")
-    model <- readRDS(file.path(model_path, "model.rds"))
-    message("model is loaded")
-
-    function(data) {
-        plant <- as.data.frame(fromJSON(data))
-        prediction <- predict(model, plant)
-        result <- as.character(prediction)
-        message(result)
-        toJSON(result)
-    }
-}

how-to-use-azureml/azureml-sdk-for-r/samples/deployment/deploy-to-local/project_files/score_new.R

@@ -1,19 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-library(jsonlite)
-
-init <- function() {
-    model_path <- Sys.getenv("AZUREML_MODEL_DIR")
-    model <- readRDS(file.path(model_path, "model.rds"))
-    message("model is loaded")
-
-    function(data) {
-        plant <- as.data.frame(fromJSON(data))
-        prediction <- predict(model, plant)
-        result <- as.character(prediction)
-        message(result)
-        message("this is a new scoring script! I was reloaded")
-        toJSON(result)
-    }
-}

how-to-use-azureml/azureml-sdk-for-r/samples/training/train-on-amlcompute/scripts/train.R

@@ -1,34 +0,0 @@
-# This script loads a dataset of which the last column is supposed to be the
-# class and logs the accuracy
-
-library(azuremlsdk)
-library(caret)
-library(optparse)
-library(datasets)
-
-
-iris_data <- data(iris)
-summary(iris_data)
-
-in_train <- createDataPartition(y = iris_data$Species, p = .8, list = FALSE)
-train_data <- iris_data[in_train,]
-test_data <- iris_data[-in_train,]
-
-# Run algorithms using 10-fold cross validation
-control <- trainControl(method = "cv", number = 10)
-metric <- "Accuracy"
-
-set.seed(7)
-model <- train(Species ~ .,
-               data = train_data,
-               method = "lda",
-               metric = metric,
-               trControl = control)
-predictions <- predict(model, test_data)
-conf_matrix <- confusionMatrix(predictions, test_data$Species)
-message(conf_matrix)
-
-log_metric_to_run(metric, conf_matrix$overall["Accuracy"])
-
-saveRDS(model, file = "./outputs/model.rds")
-message("Model saved")

how-to-use-azureml/azureml-sdk-for-r/samples/training/train-on-amlcompute/train-on-amlcompute.R

@@ -1,41 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-# Reminder: set working directory to current file location prior to running this script
-
-library(azuremlsdk)
-
-ws <- load_workspace_from_config()
-
-# Create AmlCompute cluster
-cluster_name <- "r-cluster"
-compute_target <- get_compute(ws, cluster_name = cluster_name)
-if (is.null(compute_target)) {
-    vm_size <- "STANDARD_D2_V2"
-    compute_target <- create_aml_compute(workspace = ws,
-                                       cluster_name = cluster_name,
-                                       vm_size = vm_size,
-                                       max_nodes = 1)
-
-    wait_for_provisioning_completion(compute_target, show_output = TRUE)
-}
-
-# Define estimator
-est <- estimator(source_directory = "scripts",
-                 entry_script = "train.R",
-                 compute_target = compute_target)
-
-experiment_name <- "train-r-script-on-amlcompute"
-exp <- experiment(ws, experiment_name)
-
-# Submit job and display the run details
-run <- submit_experiment(exp, est)
-view_run_details(run)
-wait_for_run_completion(run, show_output = TRUE)
-
-# Get the run metrics
-metrics <- get_run_metrics(run)
-metrics
-
-# Delete cluster
-delete_compute(compute_target)

how-to-use-azureml/azureml-sdk-for-r/samples/training/train-on-local/scripts/train.R

@@ -1,28 +0,0 @@
-# This script loads a dataset of which the last column is supposed to be the
-# class and logs the accuracy
-
-library(azuremlsdk)
-library(caret)
-library(datasets)
-
-iris_data <- data(iris)
-summary(iris_data)
-
-in_train <- createDataPartition(y = iris_data$Species, p = .8, list = FALSE)
-train_data <- iris_data[in_train,]
-test_data <- iris_data[-in_train,]
-# Run algorithms using 10-fold cross validation
-control <- trainControl(method = "cv", number = 10)
-metric <- "Accuracy"
-
-set.seed(7)
-model <- train(Species ~ .,
-               data = train_data,
-               method = "lda",
-               metric = metric,
-               trControl = control)
-predictions <- predict(model, test_data)
-conf_matrix <- confusionMatrix(predictions, test_data$Species)
-message(conf_matrix)
-
-log_metric_to_run(metric, conf_matrix$overall["Accuracy"])

how-to-use-azureml/azureml-sdk-for-r/samples/training/train-on-local/train-on-local.R

@@ -1,26 +0,0 @@
-# Copyright(c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-# Reminder: set working directory to current file location prior to running this script
-
-library(azuremlsdk)
-
-ws <- load_workspace_from_config()
-
-# Define estimator
-est <- estimator(source_directory = "scripts",
-                 entry_script = "train.R",
-                 compute_target = "local")
-
-# Initialize experiment
-experiment_name <- "train-r-script-on-local"
-exp <- experiment(ws, experiment_name)
-
-# Submit job and display the run details
-run <- submit_experiment(exp, est)
-view_run_details(run)
-wait_for_run_completion(run, show_output = TRUE)
-
-# Get the run metrics
-metrics <- get_run_metrics(run)
-metrics

how-to-use-azureml/azureml-sdk-for-r/vignettes/README.md

@@ -1,17 +0,0 @@
-## Azure Machine Learning vignettes
-
-These vignettes are end-to-end tutorials for using Azure Machine Learning SDK for R.
-
-Before running a vignette in RStudio, set the working directory to the folder that contains the vignette file (.Rmd file) in RStudio using `setwd(dirname)` or Session -> Set Working Directory -> To Source File Location. Each vignette assumes that the data and scripts are in the current working directory.
-
-The following vignettes are included:
-1. [installation](installation.Rmd): Install the Azure ML SDK for R.
-2. [configuration](configuration.Rmd): Set up an Azure ML workspace.
-3. [train-and-deploy-to-aci](train-and-deploy-to-aci): Train a caret model and deploy as a web service to Azure Container Instances (ACI).
-4. [train-with-tensorflow](train-with-tensorflow/): Train a deep learning TensorFlow model with Azure ML.
-5. [hyperparameter-tune-with-keras](hyperparameter-tune-with-keras/): Hyperparameter tune a Keras model using HyperDrive, Azure ML's hyperparameter tuning functionality.
-6. [deploy-to-aks](deploy-to-aks/): Production deploy a model as a web service to Azure Kubernetes Service (AKS).
-
-> Before you run these samples, make sure you have an Azure Machine Learning workspace. You can follow the [configuration vignette](../vignettes/configuration.Rmd) to set up a workspace. (You do not need to do this if you are running these examples on an Azure Machine Learning compute instance).
-
-For additional examples on using the R SDK, see the [samples](../samples) folder.

how-to-use-azureml/azureml-sdk-for-r/vignettes/configuration.Rmd

@@ -1,108 +0,0 @@
----
-title: "Set up an Azure ML workspace"
-date: "`r Sys.Date()`"
-output: rmarkdown::html_vignette
-vignette: >
-  %\VignetteIndexEntry{Set up an Azure ML workspace}
-  %\VignetteEngine{knitr::rmarkdown}
-  \use_package{UTF-8}
----
-
-This tutorial gets you started with the Azure Machine Learning service by walking through the requirements and instructions for setting up a workspace, the top-level resource for Azure ML.
-
-You do not need run this if you are working on an Azure Machine Learning Compute Instance, as the compute instance is already associated with an existing workspace.
-
-## What is an Azure ML workspace?
-The workspace is the top-level resource for Azure ML, providing a centralized place to work with all the artifacts you create when you use Azure ML. The workspace keeps a history of all training runs, including logs, metrics, output, and a snapshot of your scripts.
-
-When you create a new workspace, it automatically creates several Azure resources that are used by the workspace:
-
-* Azure Container Registry: Registers docker containers that you use during training and when you deploy a model. To minimize costs, ACR is lazy-loaded until deployment images are created.
-* Azure Storage account: Used as the default datastore for the workspace.
-* Azure Application Insights: Stores monitoring information about your models.
-* Azure Key Vault: Stores secrets that are used by compute targets and other sensitive information that's needed by the workspace.
-
-## Setup
-This section describes the steps required before you can access any Azure ML service functionality.
-
-### Azure subscription
-In order to create an Azure ML workspace, first you need access to an Azure subscription. An Azure subscription allows you to manage storage, compute, and other assets in the Azure cloud. You can [create a new subscription](https://azure.microsoft.com/en-us/free/) or access existing subscription information from the [Azure portal](https://portal.azure.com/). Later in this tutorial you will need information such as your subscription ID in order to create and access workspaces.
-
-### Azure ML SDK installation
-Follow the [installation guide](https://azure.github.io/azureml-sdk-for-r/articles/installation.html) to install **azuremlsdk** on your machine.
-
-## Configure your workspace
-### Workspace parameters
-To use an Azure ML workspace, you will need to supply the following information:
-
-* Your subscription ID
-* A resource group name
-* (Optional) The region that will host your workspace
-* A name for your workspace
-
-You can get your subscription ID from the [Azure portal](https://portal.azure.com/).
-
-You will also need access to a [resource group](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview#resource-groups), which organizes Azure resources and provides a default region for the resources in a group. You can see what resource groups to which you have access, or create a new one in the Azure portal. If you don't have a resource group, the `create_workspace()` method will create one for you using the name you provide.
-
-The region to host your workspace will be used if you are creating a new workspace. You do not need to specify this if you are using an existing workspace. You can find the list of supported regions [here](https://azure.microsoft.com/en-us/global-infrastructure/services/?products=machine-learning-service). You should pick a region that is close to your location or that contains your data.
-
-The name for your workspace is unique within the subscription and should be descriptive enough to discern among other workspaces. The subscription may be used only by you, or it may be used by your department or your entire enterprise, so choose a name that makes sense for your situation.
-
-The following code chunk allows you to specify your workspace parameters. It uses `Sys.getenv` to read values from environment variables, which is useful for automation. If no environment variable exists, the parameters will be set to the specified default values. Replace the default values in the code below with your default parameter values.
-
-``` {r configure_parameters, eval=FALSE}
-subscription_id <- Sys.getenv("SUBSCRIPTION_ID", unset = "<my-subscription-id>")
-resource_group <- Sys.getenv("RESOURCE_GROUP", default="<my-resource-group>")
-workspace_name <- Sys.getenv("WORKSPACE_NAME", default="<my-workspace-name>")
-workspace_region <- Sys.getenv("WORKSPACE_REGION", default="eastus2")
-``` 
-
-### Create a new workspace
-If you don't have an existing workspace and are the owner of the subscription or resource group, you can create a new workspace. If you don't have a resource group, `create_workspace()` will create one for you using the name you provide. If you don't want it to do so, set the `create_resource_group = FALSE` parameter.
-
-Note: As with other Azure services, there are limits on certain resources (e.g. AmlCompute quota) associated with the Azure ML service. Please read this [article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota.
-
-This cell will create an Azure ML workspace for you in a subscription, provided you have the correct permissions.
-
-This will fail if:
-
-* You do not have permission to create a workspace in the resource group.
-* You do not have permission to create a resource group if it does not exist.
-* You are not a subscription owner or contributor and no Azure ML workspaces have ever been created in this subscription.
-
-If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to provision the required resources.
-
-There are additional parameters that are not shown below that can be configured when creating a workspace. Please see [`create_workspace()`](https://azure.github.io/azureml-sdk-for-r/reference/create_workspace.html) for more details.
-
-``` {r create_workspace, eval=FALSE}
-library(azuremlsdk)
-
-ws <- create_workspace(name = workspace_name,
-                       subscription_id = subscription_id,
-                       resource_group = resource_group,
-                       location = workspace_region,
-                       exist_ok = TRUE)
-```
-
-You can out write out the workspace ARM properties to a config file with [`write_workspace_config()`](https://azure.github.io/azureml-sdk-for-r/reference/write_workspace_config.html). The method provides a simple way of reusing the same workspace across multiple files or projects. Users can save the workspace details with `write_workspace_config()`, and use [`load_workspace_from_config()`](https://azure.github.io/azureml-sdk-for-r/reference/load_workspace_from_config.html) to load the same workspace in different files or projects without retyping the workspace ARM properties. The method defaults to writing out the config file to the current working directory with "config.json" as the file name. To specify a different path or file name, set the `path` and `file_name` parameters.
-
-``` {r write_config, eval=FALSE}
-write_workspace_config(ws)
-```
-
-### Access an existing workspace
-You can access an existing workspace in a couple of ways. If your workspace properties were previously saved to a config file, you can load the workspace as follows:
-
-``` {r load_config, eval=FALSE}
-ws <- load_workspace_from_config()
-```
-
-If Azure ML cannot find the config file, specify the path to the config file with the `path` parameter. The method defaults to starting the search in the current directory.
-
-You can also initialize a workspace using the [`get_workspace()`](https://azure.github.io/azureml-sdk-for-r/reference/get_workspace.html) method.
-
-``` {r get_workspace, eval=FALSE}
-ws <- get_workspace(name = workspace_name,
-                    subscription_id = subscription_id,
-                    resource_group = resource_group)
-```

how-to-use-azureml/azureml-sdk-for-r/vignettes/deploy-to-aks/deploy-to-aks.Rmd

@@ -1,188 +0,0 @@
----
-title: "Deploy a web service to Azure Kubernetes Service"
-date: "`r Sys.Date()`"
-output: rmarkdown::html_vignette
-vignette: >
-  %\VignetteIndexEntry{Deploy a web service to Azure Kubernetes Service}
-  %\VignetteEngine{knitr::rmarkdown}
-  \use_package{UTF-8}
----
-
-This tutorial demonstrates how to deploy a model as a web service on [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) (AKS). AKS is good for high-scale production deployments; use it if you need one or more of the following capabilities:
-
-* Fast response time
-* Autoscaling of the deployed service
-* Hardware acceleration options such as GPU
-
-You will learn to:
-
-* Set up your testing environment
-* Register a model
-* Provision an AKS cluster
-* Deploy the model to AKS
-* Test the deployed service
-
-## Prerequisites
-If you don’t have access to an Azure ML workspace, follow the [setup tutorial](https://azure.github.io/azureml-sdk-for-r/articles/configuration.html) to configure and create a workspace.
-
-## Set up your testing environment
-Start by setting up your environment. This includes importing the **azuremlsdk** package and connecting to your workspace.
-
-### Import package
-```{r import_package, eval=FALSE}
-library(azuremlsdk)
-```
-
-### Load your workspace
-Instantiate a workspace object from your existing workspace. The following code will load the workspace details from a **config.json** file if you previously wrote one out with `write_workspace_config()`. 
-```{r load_workspace, eval=FALSE}
-ws <- load_workspace_from_config()
-```
-
-Or, you can retrieve a workspace by directly specifying your workspace details:
-```{r get_workspace, eval=FALSE}
-ws <- get_workspace("<your workspace name>", "<your subscription ID>", "<your resource group>")
-```
-
-## Register the model
-In this tutorial we will deploy a model that was trained in one of the [samples](https://github.com/Azure/azureml-sdk-for-r/blob/master/samples/training/train-on-amlcompute/train-on-amlcompute.R). The model was trained with the Iris dataset and can be used to determine if a flower is one of three Iris flower species (setosa, versicolor, virginica). We have provided the model file (`model.rds`) for the tutorial; it is located in the "project_files" directory of this vignette.
-
-First, register the model to your workspace with [`register_model()`](https://azure.github.io/azureml-sdk-for-r/reference/register_model.html). A registered model can be any collection of files, but in this case the R model file is sufficient. Azure ML will use the registered model for deployment.
-
-```{r register_model, eval=FALSE}
-model <- register_model(ws, 
-                        model_path = "project_files/model.rds", 
-                        model_name = "iris_model",
-                        description = "Predict an Iris flower type")
-```
-
-## Provision an AKS cluster
-When deploying a web service to AKS, you deploy to an AKS cluster that is connected to your workspace. There are two ways to connect an AKS cluster to your workspace:
-
-* Create the AKS cluster. The process automatically connects the cluster to the workspace. 
-* Attach an existing AKS cluster to your workspace. You can attach a cluster with the [`attach_aks_compute()`](https://azure.github.io/azureml-sdk-for-r/reference/attach_aks_compute.html) method.
-
-Creating or attaching an AKS cluster is a one-time process for your workspace. You can reuse this cluster for multiple deployments. If you delete the cluster or the resource group that contains it, you must create a new cluster the next time you need to deploy.
-
-In this tutorial, we will go with the first method of provisioning a new cluster. See the [`create_aks_compute()`](https://azure.github.io/azureml-sdk-for-r/reference/create_aks_compute.html) reference for the full set of configurable parameters. If you pick custom values for the `agent_count` and `vm_size` parameters, you need to make sure `agent_count` multiplied by `vm_size` is greater than or equal to `12` virtual CPUs.
-
-``` {r provision_cluster, eval=FALSE}
-aks_target <- create_aks_compute(ws, cluster_name = 'myakscluster')
-
-wait_for_provisioning_completion(aks_target, show_output = TRUE)
-```
-
-The Azure ML SDK does not provide support for scaling an AKS cluster. To scale the nodes in the cluster, use the UI for your AKS cluster in the Azure portal. You can only change the node count, not the VM size of the cluster.
-
-## Deploy as a web service
-### Define the inference dependencies
-To deploy a model, you need an **inference configuration**, which describes the environment needed to host the model and web service. To create an inference config, you will first need a scoring script and an Azure ML environment.
-
-The scoring script (`entry_script`) is an R script that will take as input variable values (in JSON format) and output a prediction from your model. For this tutorial, use the provided scoring file `score.R`. The scoring script must contain an `init()` method that loads your model and returns a function that uses the model to make a prediction based on the input data. See the [documentation](https://azure.github.io/azureml-sdk-for-r/reference/inference_config.html#details) for more details.
-
-Next, define an Azure ML **environment** for your script’s package dependencies. With an environment, you specify R packages (from CRAN or elsewhere) that are needed for your script to run. You can also provide the values of environment variables that your script can reference to modify its behavior. 
-
-By default Azure ML will build a default Docker image that includes R, the Azure ML SDK, and additional required dependencies for deployment. See the documentation here for the full list of dependencies that will be installed in the default container. You can also specify additional packages to be installed at runtime, or even a custom Docker image to be used instead of the base image that will be built, using the other available parameters to [`r_environment()`](https://azure.github.io/azureml-sdk-for-r/reference/r_environment.html).
-
-```{r create_env, eval=FALSE}
-r_env <- r_environment(name = "deploy_env")
-```
-
-Now you have everything you need to create an inference config for encapsulating your scoring script and environment dependencies.
-
-``` {r create_inference_config, eval=FALSE}
-inference_config <- inference_config(
-  entry_script = "score.R",
-  source_directory = "project_files",
-  environment = r_env)
-```
-
-### Deploy to AKS
-Now, define the deployment configuration that describes the compute resources needed, for example, the number of cores and memory. See the [`aks_webservice_deployment_config()`](https://azure.github.io/azureml-sdk-for-r/reference/aks_webservice_deployment_config.html) for the full set of configurable parameters.
-
-``` {r deploy_config, eval=FALSE}
-aks_config <- aks_webservice_deployment_config(cpu_cores = 1, memory_gb = 1)
-```
-
-Now, deploy your model as a web service to the AKS cluster you created earlier.
-
-```{r deploy_service, eval=FALSE}
-aks_service <- deploy_model(ws, 
-                            'my-new-aksservice', 
-                            models = list(model), 
-                            inference_config = inference_config, 
-                            deployment_config = aks_config,
-                            deployment_target = aks_target)
-
-wait_for_deployment(aks_service, show_output = TRUE)
-```
-
-To inspect the logs from the deployment:
-```{r get_logs, eval=FALSE}
-get_webservice_logs(aks_service)
-```
-
-If you encounter any issue in deploying the web service, please visit the [troubleshooting guide](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-troubleshoot-deployment).
-
-## Test the deployed service
-Now that your model is deployed as a service, you can test the service from R using [`invoke_webservice()`](https://azure.github.io/azureml-sdk-for-r/reference/invoke_webservice.html). Provide a new set of data to predict from, convert it to JSON, and send it to the service.
-
-``` {r test_service, eval=FALSE}
-library(jsonlite)
-# versicolor
-plant <- data.frame(Sepal.Length = 6.4,
-                    Sepal.Width = 2.8,
-                    Petal.Length = 4.6,
-                    Petal.Width = 1.8)
-
-# setosa
-# plant <- data.frame(Sepal.Length = 5.1,
-#                    Sepal.Width = 3.5,
-#                    Petal.Length = 1.4,
-#                    Petal.Width = 0.2)
-
-# virginica
-# plant <- data.frame(Sepal.Length = 6.7,
-#                    Sepal.Width = 3.3,
-#                    Petal.Length = 5.2,
-#                    Petal.Width = 2.3)
-
-predicted_val <- invoke_webservice(aks_service, toJSON(plant))
-message(predicted_val)
-```
-
-You can also get the web service’s HTTP endpoint, which accepts REST client calls. You can share this endpoint with anyone who wants to test the web service or integrate it into an application.
-
-``` {r eval=FALSE}
-aks_service$scoring_uri
-```
-
-## Web service authentication
-When deploying to AKS, key-based authentication is enabled by default. You can also enable token-based authentication. Token-based authentication requires clients to use an Azure Active Directory account to request an authentication token, which is used to make requests to the deployed service.
-
-To disable key-based auth, set the `auth_enabled = FALSE` parameter when creating the deployment configuration with [`aks_webservice_deployment_config()`](https://azure.github.io/azureml-sdk-for-r/reference/aks_webservice_deployment_config.html). 
-To enable token-based auth, set `token_auth_enabled = TRUE` when creating the deployment config.
-
-### Key-based authentication
-If key authentication is enabled, you can use the [`get_webservice_keys()`](https://azure.github.io/azureml-sdk-for-r/reference/get_webservice_keys.html) method to retrieve a primary and secondary authentication key. To generate a new key, use [`generate_new_webservice_key()`](https://azure.github.io/azureml-sdk-for-r/reference/generate_new_webservice_key.html).
-
-### Token-based authentication
-If token authentication is enabled, you can use the [`get_webservice_token()`](https://azure.github.io/azureml-sdk-for-r/reference/get_webservice_token.html) method to retrieve a JWT token and that token's expiration time. Make sure to request a new token after the token's expiration time.
-
-## Clean up resources
-Delete the resources once you no longer need them. Do not delete any resource you plan on still using.
-
-Delete the web service:
-```{r delete_service, eval=FALSE}
-delete_webservice(aks_service)
-```
-
-Delete the registered model:
-```{r delete_model, eval=FALSE}
-delete_model(model)
-```
-
-Delete the AKS cluster:
-```{r delete_cluster, eval=FALSE}
-delete_compute(aks_target)
-```

how-to-use-azureml/azureml-sdk-for-r/vignettes/deploy-to-aks/project_files/score.R

@@ -1,17 +0,0 @@
-#' Copyright(c) Microsoft Corporation.
-#' Licensed under the MIT license.
-
-library(jsonlite)
-
-init <- function() {
-    model_path <- Sys.getenv("AZUREML_MODEL_DIR")
-    model <- readRDS(file.path(model_path, "model.rds"))
-    message("model is loaded")
-
-    function(data) {
-        plant <- as.data.frame(fromJSON(data))
-        prediction <- predict(model, plant)
-        result <- as.character(prediction)
-        toJSON(result)
-    }
-}

how-to-use-azureml/azureml-sdk-for-r/vignettes/hyperparameter-tune-with-keras/hyperparameter-tune-with-keras.Rmd

@@ -1,242 +0,0 @@
----
-title: "Hyperparameter tune a Keras model"
-date: "`r Sys.Date()`"
-output: rmarkdown::html_vignette
-vignette: >
-  %\VignetteIndexEntry{Hyperparameter tune a Keras model}
-  %\VignetteEngine{knitr::rmarkdown}
-  \use_package{UTF-8}
----
-
-This tutorial demonstrates how you can efficiently tune hyperparameters for a model using HyperDrive, Azure ML's hyperparameter tuning functionality. You will train a Keras model on the CIFAR10 dataset, automate hyperparameter exploration, launch parallel jobs, log your results, and find the best run.
-
-### What are hyperparameters?
-
-Hyperparameters are variable parameters chosen to train a model. Learning rate, number of epochs, and batch size are all examples of hyperparameters.
-
-Using brute-force methods to find the optimal values for parameters can be time-consuming, and poor-performing runs can result in wasted money. To avoid this, HyperDrive automates hyperparameter exploration in a time-saving and cost-effective manner by launching several parallel runs with different configurations and finding the configuration that results in best performance on your primary metric.
-
-Let's get started with the example to see how it works!
-
-## Prerequisites
-
-If you don’t have access to an Azure ML workspace, follow the [setup tutorial](https://azure.github.io/azureml-sdk-for-r/articles/configuration.html) to configure and create a workspace.
-
-## Set up development environment
-The setup for your development work in this tutorial includes the following actions:
-
-* Import required packages
-* Connect to a workspace
-* Create an experiment to track your runs
-* Create a remote compute target to use for training
-
-### Import **azuremlsdk** package
-```{r eval=FALSE}
-library(azuremlsdk)
-```
-
-### Load your workspace
-Instantiate a workspace object from your existing workspace. The following code will load the workspace details from a **config.json** file if you previously wrote one out with [`write_workspace_config()`](https://azure.github.io/azureml-sdk-for-r/reference/write_workspace_config.html).
-```{r load_workpace, eval=FALSE}
-ws <- load_workspace_from_config()
-```
-
-Or, you can retrieve a workspace by directly specifying your workspace details:
-```{r get_workpace, eval=FALSE}
-ws <- get_workspace("<your workspace name>", "<your subscription ID>", "<your resource group>")
-```
-
-### Create an experiment
-An Azure ML **experiment** tracks a grouping of runs, typically from the same training script. Create an experiment to track hyperparameter tuning runs for the Keras model.
-
-```{r create_experiment, eval=FALSE}
-exp <- experiment(workspace = ws, name = 'hyperdrive-cifar10')
-```
-
-If you would like to track your runs in an existing experiment, simply specify that experiment's name to the `name` parameter of `experiment()`.
-
-### Create a compute target
-By using Azure Machine Learning Compute (AmlCompute), a managed service, data scientists can train machine learning models on clusters of Azure virtual machines. In this tutorial, you create a GPU-enabled cluster as your training environment. The code below creates the compute cluster for you if it doesn't already exist in your workspace.
-
-You may need to wait a few minutes for your compute cluster to be provisioned if it doesn't already exist.
-
-```{r create_cluster, eval=FALSE}
-cluster_name <- "gpucluster"
-
-compute_target <- get_compute(ws, cluster_name = cluster_name)
-if (is.null(compute_target))
-{
-  vm_size <- "STANDARD_NC6"
-  compute_target <- create_aml_compute(workspace = ws, 
-                                       cluster_name = cluster_name,
-                                       vm_size = vm_size, 
-                                       max_nodes = 4)
-  
-  wait_for_provisioning_completion(compute_target, show_output = TRUE)
-}
-```
-
-## Prepare the training script
-A training script called `cifar10_cnn.R` has been provided for you in the "project_files" directory of this tutorial.
-
-In order to leverage HyperDrive, the training script for your model must log the relevant metrics during model training. When you configure the hyperparameter tuning run, you specify the primary metric to use for evaluating run performance. You must log this metric so it is available to the hyperparameter tuning process.
-
-In order to log the required metrics, you need to do the following **inside the training script**:
-
-* Import the **azuremlsdk** package
-```
-library(azuremlsdk)
-```
-
-* Take the hyperparameters as command-line arguments to the script. This is necessary so that when HyperDrive carries out the hyperparameter sweep, it can run the training script with different values to the hyperparameters as defined by the search space.
-
-* Use the [`log_metric_to_run()`](https://azure.github.io/azureml-sdk-for-r/reference/log_metric_to_run.html) function to log the hyperparameters and the primary metric.
-```
-log_metric_to_run("batch_size", batch_size)
-...
-log_metric_to_run("epochs", epochs)
-...
-log_metric_to_run("lr", lr)
-...
-log_metric_to_run("decay", decay)
-...
-log_metric_to_run("Loss", results[[1]])
-```
-
-## Create an estimator
-
-An Azure ML **estimator** encapsulates the run configuration information needed for executing a training script on the compute target. Azure ML runs are run as containerized jobs on the specified compute target. By default, the Docker image built for your training job will include R, the Azure ML SDK, and a set of commonly used R packages. See the full list of default packages included [here](https://azure.github.io/azureml-sdk-for-r/reference/r_environment.html). The estimator is used to define the configuration for each of the child runs that the parent HyperDrive run will kick off.
-
-To create the estimator, define the following:
-
-* The directory that contains your scripts needed for training (`source_directory`). All the files in this directory are uploaded to the cluster node(s) for execution. The directory must contain your training script and any additional scripts required.
-* The training script that will be executed (`entry_script`).
-* The compute target (`compute_target`), in this case the AmlCompute cluster you created earlier.
-* Any environment dependencies required for training. Since the training script requires the Keras package, which is not included in the image by default, pass the package name to the `cran_packages` parameter to have it installed in the Docker container where the job will run. See the [`estimator()`](https://azure.github.io/azureml-sdk-for-r/reference/estimator.html) reference for the full set of configurable options.
-* Set the `use_gpu = TRUE` flag so the default base GPU Docker image will be built, since the job will be run on a GPU cluster.
-
-```{r create_estimator, eval=FALSE}
-est <- estimator(source_directory = "project_files",
-                 entry_script = "cifar10_cnn.R",
-                 compute_target = compute_target,
-                 cran_packages = c("keras"),
-                 use_gpu = TRUE)
-```
-
-## Configure the HyperDrive run
-To kick off hyperparameter tuning in Azure ML, you will need to configure a HyperDrive run, which will in turn launch individual children runs of the training scripts with the corresponding hyperparameter values.
-
-### Define search space
-
-In this experiment, we will use four hyperparameters: batch size, number of epochs, learning rate, and decay. In order to begin tuning, we must define the range of values we would like to explore from and how they will be distributed. This is called a parameter space definition and can be created with discrete or continuous ranges.
-
-__Discrete hyperparameters__ are specified as a choice among discrete values represented as a list.
-
-Advanced discrete hyperparameters can also be specified using a distribution. The following distributions are supported:
-
- * `quniform(low, high, q)`
- * `qloguniform(low, high, q)`
- * `qnormal(mu, sigma, q)`
- * `qlognormal(mu, sigma, q)`
-
-__Continuous hyperparameters__ are specified as a distribution over a continuous range of values. The following distributions are supported:
-
- * `uniform(low, high)`
- * `loguniform(low, high)`
- * `normal(mu, sigma)`
- * `lognormal(mu, sigma)`
-
-Here, we will use the [`random_parameter_sampling()`](https://azure.github.io/azureml-sdk-for-r/reference/random_parameter_sampling.html) function to define the search space for each hyperparameter. `batch_size` and `epochs` will be chosen from discrete sets while `lr` and `decay` will be drawn from continuous distributions.
-
-Other available sampling function options are:
-
- * [`grid_parameter_sampling()`](https://azure.github.io/azureml-sdk-for-r/reference/grid_parameter_sampling.html)
- * [`bayesian_parameter_sampling()`](https://azure.github.io/azureml-sdk-for-r/reference/bayesian_parameter_sampling.html)
-
-```{r search_space, eval=FALSE}
-sampling <- random_parameter_sampling(list(batch_size = choice(c(16, 32, 64)),
-                                           epochs = choice(c(200, 350, 500)),
-                                           lr = normal(0.0001, 0.005),
-                                           decay = uniform(1e-6, 3e-6)))
-```
-
-### Define termination policy
-
-To prevent resource waste, Azure ML can detect and terminate poorly performing runs. HyperDrive will do this automatically if you specify an early termination policy.
-
-Here, you will use the [`bandit_policy()`](https://azure.github.io/azureml-sdk-for-r/reference/bandit_policy.html), which terminates any runs where the primary metric is not within the specified slack factor with respect to the best performing training run.
-
-```{r termination_policy, eval=FALSE}
-policy <- bandit_policy(slack_factor = 0.15)
-```
-
-Other termination policy options are:
-
- * [`median_stopping_policy()`](https://azure.github.io/azureml-sdk-for-r/reference/median_stopping_policy.html)
- * [`truncation_selection_policy()`](https://azure.github.io/azureml-sdk-for-r/reference/truncation_selection_policy.html)
- 
-If no policy is provided, all runs will continue to completion regardless of performance.
-
-### Finalize configuration
-
-Now, you can create a `HyperDriveConfig` object to define your HyperDrive run. Along with the sampling and policy definitions, you need to specify the name of the primary metric that you want to track and whether we want to maximize it or minimize it. The `primary_metric_name` must correspond with the name of the primary metric you logged in your training script. `max_total_runs` specifies the total number of child runs to launch. See the [hyperdrive_config()](https://azure.github.io/azureml-sdk-for-r/reference/hyperdrive_config.html) reference for the full set of configurable parameters.
-
-```{r create_config, eval=FALSE}
-hyperdrive_config <- hyperdrive_config(hyperparameter_sampling = sampling,
-                                       primary_metric_goal("MINIMIZE"),
-                                       primary_metric_name = "Loss",
-                                       max_total_runs = 4,
-                                       policy = policy,
-                                       estimator = est)
-```
-
-## Submit the HyperDrive run
-
-Finally submit the experiment to run on your cluster. The parent HyperDrive run will launch the individual child runs. `submit_experiment()` will return a `HyperDriveRun` object that you will use to interface with the run. In this tutorial, since the cluster we created scales to a max of `4` nodes, all 4 child runs will be launched in parallel.
-
-```{r submit_run, eval=FALSE}
-hyperdrive_run <- submit_experiment(exp, hyperdrive_config)
-```
-
-You can view the HyperDrive run’s details as a table. Clicking the “Web View” link provided will bring you to Azure Machine Learning studio, where you can monitor the run in the UI.
-
-```{r eval=FALSE}
-view_run_details(hyperdrive_run)
-```
-
-Wait until hyperparameter tuning is complete before you run more code.
-
-```{r eval=FALSE}
-wait_for_run_completion(hyperdrive_run, show_output = TRUE)
-```
-
-## Analyse runs by performance
-
-Finally, you can view and compare the metrics collected during all of the child runs!
-
-```{r analyse_runs, eval=FALSE}
-# Get the metrics of all the child runs
-child_run_metrics <- get_child_run_metrics(hyperdrive_run)
-child_run_metrics
-
-# Get the child run objects sorted in descending order by the best primary metric
-child_runs <- get_child_runs_sorted_by_primary_metric(hyperdrive_run)
-child_runs
-
-# Directly get the run object of the best performing run
-best_run <- get_best_run_by_primary_metric(hyperdrive_run)
-
-# Get the metrics of the best performing run
-metrics <- get_run_metrics(best_run)
-metrics
-```
-
-The `metrics` variable will include the values of the hyperparameters that resulted in the best performing run.
-
-## Clean up resources
-Delete the resources once you no longer need them. Don't delete any resource you plan to still use. 
-
-Delete the compute cluster:
-```{r delete_compute, eval=FALSE}
-delete_compute(compute_target)
-```

how-to-use-azureml/azureml-sdk-for-r/vignettes/hyperparameter-tune-with-keras/project_files/cifar10_cnn.R

@@ -1,124 +0,0 @@
-#' Modified from: "https://github.com/rstudio/keras/blob/master/vignettes/
-#' examples/cifar10_cnn.R"
-#' 
-#' Train a simple deep CNN on the CIFAR10 small images dataset.
-#'  
-#' It gets down to 0.65 test logloss in 25 epochs, and down to 0.55 after 50
-#' epochs, though it is still underfitting at that point.
-
-library(keras)
-install_keras()
-
-library(azuremlsdk)
-
-# Parameters --------------------------------------------------------------
-
-args <- commandArgs(trailingOnly = TRUE)
-
-batch_size <- as.numeric(args[2])
-log_metric_to_run("batch_size", batch_size)
-
-epochs <- as.numeric(args[4])
-log_metric_to_run("epochs", epochs)
-
-lr <- as.numeric(args[6])
-log_metric_to_run("lr", lr)
-
-decay <- as.numeric(args[8])
-log_metric_to_run("decay", decay)
-
-data_augmentation <- TRUE
-
-
-# Data Preparation --------------------------------------------------------
-
-# See ?dataset_cifar10 for more info
-cifar10 <- dataset_cifar10()
-
-# Feature scale RGB values in test and train inputs  
-x_train <- cifar10$train$x / 255
-x_test <- cifar10$test$x / 255
-y_train <- to_categorical(cifar10$train$y, num_classes = 10)
-y_test <- to_categorical(cifar10$test$y, num_classes = 10)
-
-
-# Defining Model ----------------------------------------------------------
-
-# Initialize sequential model
-model <- keras_model_sequential()
-
-model %>%
-
-# Start with hidden 2D convolutional layer being fed 32x32 pixel images
-layer_conv_2d(
-    filter = 32, kernel_size = c(3, 3), padding = "same",
-    input_shape = c(32, 32, 3)
-  ) %>%
-  layer_activation("relu") %>%
-
-  # Second hidden layer
-  layer_conv_2d(filter = 32, kernel_size = c(3, 3)) %>%
-  layer_activation("relu") %>%
-
-  # Use max pooling
-  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
-  layer_dropout(0.25) %>%
-
-  # 2 additional hidden 2D convolutional layers
-  layer_conv_2d(filter = 32, kernel_size = c(3, 3), padding = "same") %>%
-  layer_activation("relu") %>%
-  layer_conv_2d(filter = 32, kernel_size = c(3, 3)) %>%
-  layer_activation("relu") %>%
-
-  # Use max pooling once more
-  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
-  layer_dropout(0.25) %>%
-
-  # Flatten max filtered output into feature vector 
-  # and feed into dense layer
-  layer_flatten() %>%
-  layer_dense(512) %>%
-  layer_activation("relu") %>%
-  layer_dropout(0.5) %>%
-
-  # Outputs from dense layer are projected onto 10 unit output layer
-  layer_dense(10) %>%
-  layer_activation("softmax")
-
-opt <- optimizer_rmsprop(lr, decay)
-
-model %>%
-    compile(loss = "categorical_crossentropy",
-          optimizer = opt,
-          metrics = "accuracy"
-)
-
-
-# Training ----------------------------------------------------------------
-
-if (!data_augmentation) {
-
-    model %>%
-    fit(x_train,
-        y_train,
-        batch_size = batch_size,
-        epochs = epochs,
-        validation_data = list(x_test, y_test),
-        shuffle = TRUE
-  )
-
-} else {
-
-    datagen <- image_data_generator(rotation_range = 20,
-                                  width_shift_range = 0.2,
-                                  height_shift_range = 0.2,
-                                  horizontal_flip = TRUE
-  )
-
-    datagen %>% fit_image_data_generator(x_train)
-
-    results <- evaluate(model, x_train, y_train, batch_size)
-    log_metric_to_run("Loss", results[[1]])
-    cat("Loss: ", results[[1]], "\n")
-    cat("Accuracy: ", results[[2]], "\n")
-}

how-to-use-azureml/azureml-sdk-for-r/vignettes/installation.Rmd

@@ -1,100 +0,0 @@
----
-title: "Install the Azure ML SDK for R"
-date: "`r Sys.Date()`"
-output: rmarkdown::html_vignette
-vignette: >
-  %\VignetteIndexEntry{Install the Azure ML SDK for R}
-  %\VignetteEngine{knitr::rmarkdown}
-  \use_package{UTF-8}
----
-
-This article covers the step-by-step instructions for installing the Azure ML SDK for R.
-
-You do not need run this if you are working on an Azure Machine Learning Compute Instance, as the compute instance already has the Azure ML SDK preinstalled.
-
-## Install Conda
-
-If you do not have Conda already installed on your machine, you will first need to install it, since the Azure ML R SDK uses **reticulate** to bind to the Python SDK. We recommend installing [Miniconda](https://docs.conda.io/en/latest/miniconda.html), which is a smaller, lightweight version of Anaconda. Choose the 64-bit binary for Python 3.5 or later.
-
-## Install the **azuremlsdk** R package
-You will need **remotes** to install **azuremlsdk** from the GitHub repo.
-``` {r install_remotes, eval=FALSE}
-install.packages('remotes')
-``` 
-
-Then, you can use the `install_github` function to install the package.
-``` {r install_azuremlsdk, eval=FALSE}
-remotes::install_cran('azuremlsdk', repos = 'https://cloud.r-project.org/')
-```
-
-If you are using R installed from CRAN, which comes with 32-bit and 64-bit binaries, you may need to specify the parameter `INSTALL_opts=c("--no-multiarch")` to only build for the current 64-bit architecture.
-``` {r eval=FALSE}
-remotes::install_cran('azuremlsdk', repos = 'https://cloud.r-project.org/', INSTALL_opts=c("--no-multiarch"))
-```
-
-## Install the Azure ML Python SDK
-Lastly, use the **azuremlsdk** R library to install the Python SDK. By default, `azuremlsdk::install_azureml()` will install the [latest version of the Python SDK](https://pypi.org/project/azureml-sdk/) in a conda environment called `r-azureml` if reticulate < 1.14 or `r-reticulate` if reticulate ≥ 1.14.
-``` {r install_pythonsdk, eval=FALSE}
-azuremlsdk::install_azureml()
-```
-
-If you would like to override the default version, environment name, or Python version, you can pass in those arguments. If you would like to restart the R session after installation or delete the conda environment if it already exists and create a new environment, you can also do so: 
-``` {r eval=FALSE}
-azuremlsdk::install_azureml(version = NULL, 
-                            custom_envname = "<your conda environment name>",
-                            conda_python_version = "<desired python version>",
-                            restart_session = TRUE,
-                            remove_existing_env = TRUE)
-```
-
-## Test installation
-You can confirm your installation worked by loading the library and successfully retrieving a run.
-``` {r test_installation, eval=FALSE}
-library(azuremlsdk)
-get_current_run()
-```
-
-## Troubleshooting
--   In step 3 of the installation, if you get ssl errors on windows, it is due to an
-outdated openssl binary. Install the latest openssl binaries from
-[here](https://wiki.openssl.org/index.php/Binaries).
-
-- If installation fails due to this error:
-
-  ```R
-  Error in strptime(xx, f, tz = tz) : 
-    (converted from warning) unable to identify current timezone 'C':
-  please set environment variable 'TZ'
-  In R CMD INSTALL
-  Error in i.p(...) : 
-    (converted from warning) installation of package ‘C:/.../azureml_0.4.0.tar.gz’ had non-zero exit
-    status
-  ```
-  
-  You will need to set your time zone environment variable to GMT and restart the installation process.
-  
-  ```R
-  Sys.setenv(TZ='GMT')
-  ```
-
-- If the following permission error occurs while installing in RStudio,
-  change your RStudio session to administrator mode, and re-run the installation command.
-
-  ```R
-  Downloading GitHub repo Azure/azureml-sdk-for-r@master
-  Skipping 2 packages ahead of CRAN: reticulate, rlang
-  Running `R CMD build`...
-  
-  Error: (converted from warning) invalid package
-  'C:/.../file2b441bf23631'
-  In R CMD INSTALL
-  Error in i.p(...) : 
-    (converted from warning) installation of package
-    ‘C:/.../file2b441bf23631’ had non-zero exit status
-  In addition: Warning messages:
-  1: In file(con, "r") :
-    cannot open file 'C:...\file2b44144a540f': Permission denied
-  2: In file(con, "r") :
-    cannot open file 'C:...\file2b4463c21577': Permission denied
-  ```
-  

how-to-use-azureml/azureml-sdk-for-r/vignettes/train-and-deploy-to-aci/project_files/accident_predict.R

@@ -1,16 +0,0 @@
-#' Copyright(c) Microsoft Corporation.
-#' Licensed under the MIT license.
-
-library(jsonlite)
-
-init <- function() {
-    model_path <- Sys.getenv("AZUREML_MODEL_DIR")
-    model <- readRDS(file.path(model_path, "model.rds"))
-    message("logistic regression model loaded")
-
-    function(data) {
-        vars <- as.data.frame(fromJSON(data))
-        prediction <- as.numeric(predict(model, vars, type = "response") * 100)
-        toJSON(prediction)
-    }
-}

how-to-use-azureml/azureml-sdk-for-r/vignettes/train-and-deploy-to-aci/project_files/accidents.R

@@ -1,33 +0,0 @@
-#' Copyright(c) Microsoft Corporation.
-#' Licensed under the MIT license.
-
-library(azuremlsdk)
-library(optparse)
-library(caret)
-
-options <- list(
-  make_option(c("-d", "--data_folder"))
-)
-
-opt_parser <- OptionParser(option_list = options)
-opt <- parse_args(opt_parser)
-
-paste(opt$data_folder)
-
-accidents <- readRDS(file.path(opt$data_folder, "accidents.Rd"))
-summary(accidents)
-
-mod <- glm(dead ~ dvcat + seatbelt + frontal + sex + ageOFocc + yearVeh + airbag + occRole, family = binomial, data = accidents)
-summary(mod)
-predictions <- factor(ifelse(predict(mod) > 0.1, "dead", "alive"))
-conf_matrix <- confusionMatrix(predictions, accidents$dead)
-message(conf_matrix)
-
-log_metric_to_run("Accuracy", conf_matrix$overall["Accuracy"])
-
-output_dir = "outputs"
-if (!dir.exists(output_dir)) {
-    dir.create(output_dir)
-}
-saveRDS(mod, file = "./outputs/model.rds")
-message("Model saved")

how-to-use-azureml/azureml-sdk-for-r/vignettes/train-and-deploy-to-aci/train-and-deploy-to-aci.Rmd

@@ -1,326 +0,0 @@
----
-title: "Train and deploy your first model with Azure ML"
-author: "David Smith"
-date: "`r Sys.Date()`"
-output: rmarkdown::html_vignette
-vignette: >
-  %\VignetteIndexEntry{Train and deploy your first model with Azure ML}
-  %\VignetteEngine{knitr::rmarkdown}
-  \use_package{UTF-8}
----
-
-In this tutorial, you learn the foundational design patterns in Azure Machine Learning.  You'll train and deploy a **caret** model to predict the likelihood of a fatality in an automobile accident. After completing this tutorial, you'll have the practical knowledge of the R SDK to scale up to developing more-complex experiments and workflows.
-
-In this tutorial, you learn the following tasks:
-
-* Connect your workspace
-* Load data and prepare for training
-* Upload data to the datastore so it is available for remote training
-* Create a compute resource
-* Train a caret model to predict probability of fatality
-* Deploy a prediction endpoint
-* Test the model from R
-
-## Prerequisites
-
-If you don't have access to an Azure ML workspace, follow the [setup tutorial](https://azure.github.io/azureml-sdk-for-r/articles/configuration.html) to configure and create a workspace.
-
-## Set up your development environment
-The setup for your development work in this tutorial includes the following actions:
-
-* Install required packages
-* Connect to a workspace, so that your local computer can communicate with remote resources
-* Create an experiment to track your runs
-* Create a remote compute target to use for training
-
-### Install required packages
-This tutorial assumes you already have the Azure ML SDK installed. Go ahead and import the **azuremlsdk** package.
-
-```{r eval=FALSE}
-library(azuremlsdk)
-```
-
-The tutorial uses data from the [**DAAG** package](https://cran.r-project.org/package=DAAG). Install the package if you don't have it.
-
-```{r eval=FALSE}
-install.packages("DAAG")
-```
-
-The training and scoring scripts (`accidents.R` and `accident_predict.R`) have some additional dependencies. If you plan on running those scripts locally, make sure you have those required packages as well.
-
-### Load your workspace
-Instantiate a workspace object from your existing workspace. The following code will load the workspace details from the **config.json** file. You can also retrieve a workspace using [`get_workspace()`](https://azure.github.io/azureml-sdk-for-r/reference/get_workspace.html).
-
-```{r load_workpace, eval=FALSE}
-ws <- load_workspace_from_config()
-```
-
-### Create an experiment
-An Azure ML experiment tracks a grouping of runs, typically from the same training script. Create an experiment to track the runs for training the caret model on the accidents data.
-
-```{r create_experiment, eval=FALSE}
-experiment_name <- "accident-logreg"
-exp <- experiment(ws, experiment_name)
-```
-
-### Create a compute target
-By using Azure Machine Learning Compute (AmlCompute), a managed service, data scientists can train machine learning models on clusters of Azure virtual machines. Examples include VMs with GPU support. In this tutorial, you create a single-node AmlCompute cluster as your training environment. The code below creates the compute cluster for you if it doesn't already exist in your workspace.
-
-You may need to wait a few minutes for your compute cluster to be provisioned if it doesn't already exist.
-
-```{r create_cluster, eval=FALSE}
-cluster_name <- "rcluster"
-compute_target <- get_compute(ws, cluster_name = cluster_name)
-if (is.null(compute_target)) {
-  vm_size <- "STANDARD_D2_V2" 
-  compute_target <- create_aml_compute(workspace = ws,
-                                       cluster_name = cluster_name,
-                                       vm_size = vm_size,
-                                       max_nodes = 1)
-  
-  wait_for_provisioning_completion(compute_target, show_output = TRUE)
-}
-```
-
-## Prepare data for training
-This tutorial uses data from the **DAAG** package. This dataset includes data from over 25,000 car crashes in the US, with variables you can use to predict the likelihood of a fatality. First, import the data into R and transform it into a new dataframe `accidents` for analysis, and export it to an `Rdata` file.
-
-```{r load_data, eval=FALSE}
-library(DAAG)
-data(nassCDS)
-
-accidents <- na.omit(nassCDS[,c("dead","dvcat","seatbelt","frontal","sex","ageOFocc","yearVeh","airbag","occRole")])
-accidents$frontal <- factor(accidents$frontal, labels=c("notfrontal","frontal"))
-accidents$occRole <- factor(accidents$occRole)
-
-saveRDS(accidents, file="accidents.Rd")
-```
-
-### Upload data to the datastore
-Upload data to the cloud so that it can be access by your remote training environment. Each Azure ML workspace comes with a default datastore that stores the connection information to the Azure blob container that is provisioned in the storage account attached to the workspace. The following code will upload the accidents data you created above to that datastore.
-
-```{r upload_data, eval=FALSE}
-ds <- get_default_datastore(ws)
-
-target_path <- "accidentdata"
-upload_files_to_datastore(ds,
-                          list("./project_files/accidents.Rd"),
-                          target_path = target_path,
-                          overwrite = TRUE)
-```
-
-
-## Train a model
-
-For this tutorial, fit a logistic regression model on your uploaded data using your remote compute cluster. To submit a job, you need to:
-
-* Prepare the training script
-* Create an estimator
-* Submit the job
-
-### Prepare the training script
-A training script called `accidents.R` has been provided for you in the "project_files" directory of this tutorial. Notice the following details **inside the training script** that have been done to leverage the Azure ML service for training:
-
-* The training script takes an argument `-d` to find the directory that contains the training data. When you define and submit your job later, you point to the datastore for this argument. Azure ML will mount the storage folder to the remote cluster for the training job.
-* The training script logs the final accuracy as a metric to the run record in Azure ML using `log_metric_to_run()`. The Azure ML SDK provides a set of logging APIs for logging various metrics during training runs. These metrics are recorded and persisted in the experiment run record. The metrics can then be accessed at any time or viewed in the run details page in [Azure Machine Learning studio](http://ml.azure.com). See the [reference](https://azure.github.io/azureml-sdk-for-r/reference/index.html#section-training-experimentation) for the full set of logging methods `log_*()`.
-* The training script saves your model into a directory named **outputs**. The `./outputs` folder receives special treatment by Azure ML. During training, files written to `./outputs` are automatically uploaded to your run record by Azure ML and persisted as artifacts. By saving the trained model to `./outputs`, you'll be able to access and retrieve your model file even after the run is over and you no longer have access to your remote training environment.
-
-### Create an estimator
-
-An Azure ML estimator encapsulates the run configuration information needed for executing a training script on the compute target. Azure ML runs are run as containerized jobs on the specified compute target. By default, the Docker image built for your training job will include R, the Azure ML SDK, and a set of commonly used R packages. See the full list of default packages included [here](https://azure.github.io/azureml-sdk-for-r/reference/r_environment.html).
-
-To create the estimator, define:
-
-* The directory that contains your scripts needed for training (`source_directory`). All the files in this directory are uploaded to the cluster node(s) for execution. The directory must contain your training script and any additional scripts required.
-* The training script that will be executed (`entry_script`).
-* The compute target (`compute_target`), in this case the AmlCompute cluster you created earlier.
-* The parameters required from the training script (`script_params`). Azure ML will run your training script as a command-line script with `Rscript`. In this tutorial you specify one argument to the script, the data directory mounting point, which you can access with `ds$path(target_path)`.
-* Any environment dependencies required for training. The default Docker image built for training already contains the three packages (`caret`, `e1071`, and `optparse`) needed in the training script.  So you don't need to specify additional information. If you are using R packages that are not included by default, use the estimator's `cran_packages` parameter to add additional CRAN packages. See the [`estimator()`](https://azure.github.io/azureml-sdk-for-r/reference/estimator.html) reference for the full set of configurable options.
-
-```{r create_estimator, eval=FALSE}
-est <- estimator(source_directory = "project_files",
-                 entry_script = "accidents.R",
-                 script_params = list("--data_folder" = ds$path(target_path)),
-                 compute_target = compute_target
-                 )
-```
-
-### Submit the job on the remote cluster
-
-Finally submit the job to run on your cluster. `submit_experiment()` returns a Run object that you then use to interface with the run. In total, the first run takes **about 10 minutes**. But for later runs, the same Docker image is reused as long as the script dependencies don't change.  In this case, the image is cached and the container startup time is much faster.
-
-```{r submit_job, eval=FALSE}
-run <- submit_experiment(exp, est)
-```
-
-You can view a table of the run's details. Clicking the "Web View" link provided will bring you to Azure Machine Learning studio, where you can monitor the run in the UI.
-
-```{r view_run, eval=FALSE}
-view_run_details(run)
-```
-
-Model training happens in the background. Wait until the model has finished training before you run more code.
-
-```{r wait_run, eval=FALSE}
-wait_for_run_completion(run, show_output = TRUE)
-```
-
-You -- and colleagues with access to the workspace -- can submit multiple experiments in parallel, and Azure ML will take of scheduling the tasks on the compute cluster. You can even configure the cluster to automatically scale up to multiple nodes, and scale back when there are no more compute tasks in the queue. This configuration is a cost-effective way for teams to share compute resources.
-
-## Retrieve training results
-Once your model has finished training, you can access the artifacts of your job that were persisted to the run record, including any metrics logged and the final trained model.
-
-### Get the logged metrics
-In the training script `accidents.R`, you logged a metric from your model: the accuracy of the predictions in the training data. You can see metrics in the [studio](https://ml.azure.com), or extract them to the local session as an R list as follows:
-
-```{r metrics, eval=FALSE}
-metrics <- get_run_metrics(run)
-metrics
-```
-
-If you've run multiple experiments (say, using differing variables, algorithms, or hyperparamers), you can use the metrics from each run to compare and choose the model you'll use in production.
-
-### Get the trained model
-You can retrieve the trained model and look at the results in your local R session. The following code will download the contents of the `./outputs` directory, which includes the model file.
-
-```{r retrieve_model, eval=FALSE}
-download_files_from_run(run, prefix="outputs/")
-accident_model <- readRDS("project_files/outputs/model.rds")
-summary(accident_model)
-```
-
-You see some factors that contribute to an increase in the estimated probability of death:
-
-* higher impact speed 
-* male driver
-* older occupant
-* passenger
-
-You see lower probabilities of death with:
-
-* presence of airbags
-* presence seatbelts
-* frontal collision 
-
-The vehicle year of manufacture does not have a significant effect.
-
-You can use this model to make new predictions:
-
-```{r manual_predict, eval=FALSE}
-newdata <- data.frame( # valid values shown below
- dvcat="10-24",        # "1-9km/h" "10-24"   "25-39"   "40-54"   "55+"  
- seatbelt="none",      # "none"   "belted"  
- frontal="frontal",    # "notfrontal" "frontal"
- sex="f",              # "f" "m"
- ageOFocc=16,          # age in years, 16-97
- yearVeh=2002,         # year of vehicle, 1955-2003
- airbag="none",        # "none"   "airbag"   
- occRole="pass"        # "driver" "pass"
- )
-
-## predicted probability of death for these variables, as a percentage
-as.numeric(predict(accident_model,newdata, type="response")*100)
-```
-
-## Deploy as a web service
-
-With your model, you can predict the danger of death from a collision. Use Azure ML to deploy your model as a prediction service. In this tutorial, you will deploy the web service in [Azure Container Instances](https://docs.microsoft.com/en-us/azure/container-instances/) (ACI).
-
-### Register the model
-
-First, register the model you downloaded to your workspace with [`register_model()`](https://azure.github.io/azureml-sdk-for-r/reference/register_model.html). A registered model can be any collection of files, but in this case the R model object is sufficient. Azure ML will use the registered model for deployment.
-
-```{r register_model, eval=FALSE}
-model <- register_model(ws, 
-                        model_path = "project_files/outputs/model.rds", 
-                        model_name = "accidents_model",
-                        description = "Predict probablity of auto accident")
-```
-
-### Define the inference dependencies
-To create a web service for your model, you first need to create a scoring script (`entry_script`), an R script that will take as input variable values (in JSON format) and output a prediction from your model. For this tutorial, use the provided scoring file `accident_predict.R`. The scoring script must contain an `init()` method that loads your model and returns a function that uses the model to make a prediction based on the input data. See the [documentation](https://azure.github.io/azureml-sdk-for-r/reference/inference_config.html#details) for more details.
-
-Next, define an Azure ML **environment** for your script's package dependencies. With an environment, you specify R packages (from CRAN or elsewhere) that are needed for your script to run. You can also provide the values of environment variables that your script can reference to modify its behavior. By default, Azure ML will build the same default Docker image used with the estimator for training. Since the tutorial has no special requirements, create an environment with no special attributes.
-
-```{r create_environment, eval=FALSE}
-r_env <- r_environment(name = "basic_env")
-```
-
-If you want to use your own Docker image for deployment instead, specify the `custom_docker_image` parameter. See the [`r_environment()`](https://azure.github.io/azureml-sdk-for-r/reference/r_environment.html) reference for the full set of configurable options for defining an environment.
-
-Now you have everything you need to create an **inference config** for encapsulating your scoring script and environment dependencies.
-
-``` {r create_inference_config, eval=FALSE}
-inference_config <- inference_config(
-  entry_script = "accident_predict.R",
-  source_directory = "project_files",
-  environment = r_env)
-```
-
-### Deploy to ACI
-In this tutorial, you will deploy your service to ACI. This code provisions a single container to respond to inbound requests, which is suitable for testing and light loads. See [`aci_webservice_deployment_config()`](https://azure.github.io/azureml-sdk-for-r/reference/aci_webservice_deployment_config.html) for additional configurable options. (For production-scale deployments, you can also [deploy to Azure Kubernetes Service](https://azure.github.io/azureml-sdk-for-r/articles/deploy-to-aks/deploy-to-aks.html).)
-
-``` {r create_aci_config, eval=FALSE}
-aci_config <- aci_webservice_deployment_config(cpu_cores = 1, memory_gb = 0.5)
-```
-
-Now you deploy your model as a web service. Deployment **can take several minutes**. 
-
-```{r deploy_service, eval=FALSE}
-aci_service <- deploy_model(ws, 
-                            'accident-pred', 
-                            list(model), 
-                            inference_config, 
-                            aci_config)
-
-wait_for_deployment(aci_service, show_output = TRUE)
-```
-
-If you encounter any issue in deploying the web service, please visit the [troubleshooting guide](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-troubleshoot-deployment).
-
-## Test the deployed service
-
-Now that your model is deployed as a service, you can test the service from R using [`invoke_webservice()`](https://azure.github.io/azureml-sdk-for-r/reference/invoke_webservice.html).  Provide a new set of data to predict from, convert it to JSON, and send it to the service.
-
-```{r test_deployment, eval=FALSE}
-library(jsonlite)
-
-newdata <- data.frame( # valid values shown below
- dvcat="10-24",        # "1-9km/h" "10-24"   "25-39"   "40-54"   "55+"  
- seatbelt="none",      # "none"   "belted"  
- frontal="frontal",    # "notfrontal" "frontal"
- sex="f",              # "f" "m"
- ageOFocc=22,          # age in years, 16-97
- yearVeh=2002,         # year of vehicle, 1955-2003
- airbag="none",        # "none"   "airbag"   
- occRole="pass"        # "driver" "pass"
- )
-
-prob <- invoke_webservice(aci_service, toJSON(newdata))
-prob
-```
-
-You can also get the web service's HTTP endpoint, which accepts REST client calls. You can share this endpoint with anyone who wants to test the web service or integrate it into an application.
-
-```{r get_endpoint, eval=FALSE}
-aci_service$scoring_uri
-```
-
-## Clean up resources
-
-Delete the resources once you no longer need them. Don't delete any resource you plan to still use. 
-
-Delete the web service:
-```{r delete_service, eval=FALSE}
-delete_webservice(aci_service)
-```
-
-Delete the registered model:
-```{r delete_model, eval=FALSE}
-delete_model(model)
-```
-
-Delete the compute cluster:
-```{r delete_compute, eval=FALSE}
-delete_compute(compute_target)
-```

how-to-use-azureml/azureml-sdk-for-r/vignettes/train-with-tensorflow/project_files/tf_mnist.R

@@ -1,62 +0,0 @@
-# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
-# Copyright 2016 RStudio, Inc. All Rights Reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# ==============================================================================
-
-
-library(tensorflow)
-install_tensorflow(version = "1.13.2-gpu")
-
-library(azuremlsdk)
-
-# Create the model
-x <- tf$placeholder(tf$float32, shape(NULL, 784L))
-W <- tf$Variable(tf$zeros(shape(784L, 10L)))
-b <- tf$Variable(tf$zeros(shape(10L)))
-
-y <- tf$nn$softmax(tf$matmul(x, W) + b)
-
-# Define loss and optimizer
-y_ <- tf$placeholder(tf$float32, shape(NULL, 10L))
-cross_entropy <- tf$reduce_mean(-tf$reduce_sum(y_ * log(y),
-                                               reduction_indices = 1L))
-train_step <- tf$train$GradientDescentOptimizer(0.5)$minimize(cross_entropy)
-
-# Create session and initialize  variables
-sess <- tf$Session()
-sess$run(tf$global_variables_initializer())
-
-# Load mnist data    )
-datasets <- tf$contrib$learn$datasets
-mnist <- datasets$mnist$read_data_sets("MNIST-data", one_hot = TRUE)
-
-# Train
-for (i in 1:1000) {
-    batches <- mnist$train$next_batch(100L)
-    batch_xs <- batches[[1]]
-    batch_ys <- batches[[2]]
-    sess$run(train_step,
-           feed_dict = dict(x = batch_xs, y_ = batch_ys))
-}
-
-# Test trained model
-correct_prediction <- tf$equal(tf$argmax(y, 1L), tf$argmax(y_, 1L))
-accuracy <- tf$reduce_mean(tf$cast(correct_prediction, tf$float32))
-cat("Accuracy: ", sess$run(accuracy,
-                           feed_dict = dict(x = mnist$test$images,
-                                            y_ = mnist$test$labels)))
-
-log_metric_to_run("accuracy",
-                  sess$run(accuracy, feed_dict = dict(x = mnist$test$images,
-                                                      y_ = mnist$test$labels)))

how-to-use-azureml/azureml-sdk-for-r/vignettes/train-with-tensorflow/train-with-tensorflow.Rmd

@@ -1,143 +0,0 @@
----
-title: "Train a TensorFlow model"
-date: "`r Sys.Date()`"
-output: rmarkdown::html_vignette
-vignette: >
-  %\VignetteIndexEntry{Train a TensorFlow model}
-  %\VignetteEngine{knitr::rmarkdown}
-  \use_package{UTF-8}
----
-
-This tutorial demonstrates how run a TensorFlow job at scale using Azure ML. You will train a TensorFlow model to classify handwritten digits (MNIST) using a deep neural network (DNN) and log your results to the Azure ML service.
-
-## Prerequisites
-If you don’t have access to an Azure ML workspace, follow the [setup tutorial](https://azure.github.io/azureml-sdk-for-r/articles/configuration.html) to configure and create a workspace.
-
-## Set up development environment
-The setup for your development work in this tutorial includes the following actions:
-
-* Import required packages
-* Connect to a workspace
-* Create an experiment to track your runs
-* Create a remote compute target to use for training
-
-### Import **azuremlsdk** package
-```{r eval=FALSE}
-library(azuremlsdk)
-```
-
-### Load your workspace
-Instantiate a workspace object from your existing workspace. The following code will load the workspace details from a **config.json** file if you previously wrote one out with [`write_workspace_config()`](https://azure.github.io/azureml-sdk-for-r/reference/write_workspace_config.html).
-```{r load_workpace, eval=FALSE}
-ws <- load_workspace_from_config()
-```
-
-Or, you can retrieve a workspace by directly specifying your workspace details:
-```{r get_workpace, eval=FALSE}
-ws <- get_workspace("<your workspace name>", "<your subscription ID>", "<your resource group>")
-```
-
-### Create an experiment
-An Azure ML **experiment** tracks a grouping of runs, typically from the same training script. Create an experiment to track the runs for training the TensorFlow model on the MNIST data.
-
-```{r create_experiment, eval=FALSE}
-exp <- experiment(workspace = ws, name = "tf-mnist")
-```
-
-If you would like to track your runs in an existing experiment, simply specify that experiment's name to the `name` parameter of `experiment()`.
-
-### Create a compute target
-By using Azure Machine Learning Compute (AmlCompute), a managed service, data scientists can train machine learning models on clusters of Azure virtual machines. In this tutorial, you create a GPU-enabled cluster as your training environment. The code below creates the compute cluster for you if it doesn't already exist in your workspace.
-
-You may need to wait a few minutes for your compute cluster to be provisioned if it doesn't already exist.
-
-```{r create_cluster, eval=FALSE}
-cluster_name <- "gpucluster"
-compute_target <- get_compute(ws, cluster_name = cluster_name)
-if (is.null(compute_target))
-{
-  vm_size <- "STANDARD_NC6"
-  compute_target <- create_aml_compute(workspace = ws, 
-                                       cluster_name = cluster_name,
-                                       vm_size = vm_size, 
-                                       max_nodes = 4)
-  
-  wait_for_provisioning_completion(compute_target, show_output = TRUE)
-}
-```
-
-## Prepare the training script
-
-A training script called `tf_mnist.R` has been provided for you in the "project_files" directory of this tutorial. The Azure ML SDK provides a set of logging APIs for logging various metrics during training runs. These metrics are recorded and persisted in the experiment run record, and can be be accessed at any time or viewed in the run details page in [Azure Machine Learning studio](http://ml.azure.com/).
-
-In order to collect and upload run metrics, you need to do the following **inside the training script**:
-
-* Import the **azuremlsdk** package
-```
-library(azuremlsdk)
-```
-
-* Add the [`log_metric_to_run()`](https://azure.github.io/azureml-sdk-for-r/reference/log_metric_to_run.html) function to track our primary metric,  "accuracy", for this experiment. If you have your own training script with several important metrics, simply create a logging call for each one within the script.
-```
-log_metric_to_run("accuracy",
-                  sess$run(accuracy,
-                  feed_dict = dict(x = mnist$test$images, y_ = mnist$test$labels)))
-```
-
-See the [reference](https://azure.github.io/azureml-sdk-for-r/reference/index.html#section-training-experimentation) for the full set of logging methods `log_*()` available from the R SDK.
-
-## Create an estimator
-
-An Azure ML **estimator** encapsulates the run configuration information needed for executing a training script on the compute target. Azure ML runs are run as containerized jobs on the specified compute target. By default, the Docker image built for your training job will include R, the Azure ML SDK, and a set of commonly used R packages. See the full list of default packages included [here](https://azure.github.io/azureml-sdk-for-r/reference/r_environment.html).
-
-To create the estimator, define the following:
-
-* The directory that contains your scripts needed for training (`source_directory`). All the files in this directory are uploaded to the cluster node(s) for execution. The directory must contain your training script and any additional scripts required.
-* The training script that will be executed (`entry_script`).
-* The compute target (`compute_target`), in this case the AmlCompute cluster you created earlier.
-* Any environment dependencies required for training. Since the training script requires the TensorFlow package, which is not included in the image by default, pass the package name to the `cran_packages` parameter to have it installed in the Docker container where the job will run. See the [`estimator()`](https://azure.github.io/azureml-sdk-for-r/reference/estimator.html) reference for the full set of configurable options.
-* Set the `use_gpu = TRUE` flag so the default base GPU Docker image will be built, since the job will be run on a GPU cluster.
-
-```{r create_estimator, eval=FALSE}
-est <- estimator(source_directory = "project_files",
-                 entry_script = "tf_mnist.R",
-                 compute_target = compute_target,
-                 cran_packages = c("tensorflow"),
-                 use_gpu = TRUE)
-```
-
-## Submit the job
-
-Finally submit the job to run on your cluster. [`submit_experiment()`](https://azure.github.io/azureml-sdk-for-r/reference/submit_experiment.html) returns a `Run` object that you can then use to interface with the run.
-
-```{r submit_job, eval=FALSE}
-run <- submit_experiment(exp, est)
-```
-
-You can view the run’s details as a table. Clicking the “Web View” link provided will bring you to Azure Machine Learning studio, where you can monitor the run in the UI.
-
-```{r eval=FALSE}
-view_run_details(run)
-```
-
-Model training happens in the background. Wait until the model has finished training before you run more code.
-
-```{r eval=FALSE}
-wait_for_run_completion(run, show_output = TRUE)
-```
-
-## View run metrics
-Once your job has finished, you can view the metrics collected during your TensorFlow run.
-
-```{r get_metrics, eval=FALSE}
-metrics <- get_run_metrics(run)
-metrics
-```
-
-## Clean up resources
-Delete the resources once you no longer need them. Don't delete any resource you plan to still use. 
-
-Delete the compute cluster:
-```{r delete_compute, eval=FALSE}
-delete_compute(compute_target)
-```

how-to-use-azureml/deployment/production-deploy-to-aks/production-deploy-to-aks.ipynb

@@ -334,14 +334,27 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# Use the default configuration (can also provide parameters to customize)\n",
-        "prov_config = AksCompute.provisioning_configuration()\n",
+        "from azureml.core.compute import ComputeTarget\n",
+        "from azureml.core.compute_target import ComputeTargetException\n",
         "\n",
+        "# Choose a name for your AKS cluster\n",
         "aks_name = 'my-aks-9' \n",
-        "# Create the cluster\n",
-        "aks_target = ComputeTarget.create(workspace = ws, \n",
-        "                                  name = aks_name, \n",
-        "                                  provisioning_configuration = prov_config)"
+        "\n",
+        "# Verify that cluster does not exist already\n",
+        "try:\n",
+        "    aks_target = ComputeTarget(workspace=ws, name=aks_name)\n",
+        "    print('Found existing cluster, use it.')\n",
+        "except ComputeTargetException:\n",
+        "    # Use the default configuration (can also provide parameters to customize)\n",
+        "    prov_config = AksCompute.provisioning_configuration()\n",
+        "\n",
+        "    # Create the cluster\n",
+        "    aks_target = ComputeTarget.create(workspace = ws, \n",
+        "                                    name = aks_name, \n",
+        "                                    provisioning_configuration = prov_config)\n",
+        "\n",
+        "if aks_target.get_status() != \"Succeeded\":\n",
+        "    aks_target.wait_for_completion(show_output=True)"
       ]
     },
     {

how-to-use-azureml/explain-model/azure-integration/remote-explanation/explain-model-on-amlcompute.yml

@@ -6,6 +6,6 @@ dependencies:
   - interpret-community[visualization]
   - matplotlib
   - azureml-contrib-interpret
-  - sklearn-pandas
+  - sklearn-pandas<2.0.0
   - azureml-dataset-runtime
   - ipywidgets

how-to-use-azureml/explain-model/azure-integration/scoring-time/train-explain-model-locally-and-deploy.yml

@@ -6,5 +6,5 @@ dependencies:
   - interpret-community[visualization]
   - matplotlib
   - azureml-contrib-interpret
-  - sklearn-pandas
+  - sklearn-pandas<2.0.0
   - ipywidgets

how-to-use-azureml/explain-model/azure-integration/scoring-time/train-explain-model-on-amlcompute-and-deploy.yml

@@ -6,7 +6,7 @@ dependencies:
   - interpret-community[visualization]
   - matplotlib
   - azureml-contrib-interpret
-  - sklearn-pandas
+  - sklearn-pandas<2.0.0
   - azureml-dataset-runtime
   - azureml-core
   - ipywidgets

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-data-transfer.ipynb

@@ -34,7 +34,7 @@
         "| Azure Data Lake Storage Gen 1 | Yes | Yes |\n",
         "| Azure Data Lake Storage Gen 2 | Yes | Yes |\n",
         "| Azure SQL Database | Yes | Yes |\n",
-        "| Azure Database for PostgreSQL | Yes | Yes |",
+        "| Azure Database for PostgreSQL | Yes | Yes |\n",
         "| Azure Database for MySQL | Yes | Yes |"
       ]
     },
@@ -558,7 +558,7 @@
   "metadata": {
     "authors": [
       {
-        "name": "sanpil"
+        "name": "shbijlan"
       }
     ],
     "category": "tutorial",

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-how-to-use-estimatorstep.ipynb

@@ -100,7 +100,7 @@
         "from azureml.core.compute_target import ComputeTargetException\n",
         "\n",
         "# choose a name for your cluster\n",
-        "cluster_name = \"cpu-cluster\"\n",
+        "cluster_name = \"amlcomp\"\n",
         "\n",
         "try:\n",
         "    cpu_cluster = ComputeTarget(workspace=ws, name=cluster_name)\n",
@@ -147,10 +147,8 @@
         "    'script_params' accepts a dictionary. However 'estimator_entry_script_arguments' parameter expects arguments as\n",
         "    a list.\n",
         "\n",
-        "> Estimator object initialization involves specifying a list of DataReference objects in its 'inputs' parameter.\n",
-        "    In Pipelines, a step can take another step's output or DataReferences as input. So when creating an EstimatorStep,\n",
-        "    the parameters 'inputs' and 'outputs' need to be set explicitly and that will override 'inputs' parameter\n",
-        "    specified in the Estimator object.\n",
+        "> Estimator object initialization involves specifying a list of data input and output.\n",
+        "    In Pipelines, a step can take another step's output as input. So when creating an EstimatorStep.\n",
         "   \n",
         "> The best practice is to use separate folders for scripts and its dependent files for each step and specify that folder as the `source_directory` for the step. This helps reduce the size of the snapshot created for the step (only the specific folder is snapshotted). Since changes in any files in the `source_directory` would trigger a re-upload of the snapshot, this helps keep the reuse of the step when there are no changes in the `source_directory` of the step."
       ]
@@ -166,17 +164,27 @@
       "outputs": [],
       "source": [
         "from azureml.core import Datastore\n",
-        "from azureml.data.data_reference import DataReference\n",
-        "from azureml.pipeline.core import PipelineData\n",
         "\n",
         "def_blob_store = Datastore(ws, \"workspaceblobstore\")\n",
         "\n",
-        "input_data = DataReference(\n",
-        "    datastore=def_blob_store,\n",
-        "    data_reference_name=\"input_data\",\n",
-        "    path_on_datastore=\"20newsgroups/20news.pkl\")\n",
+        "#upload input data to workspaceblobstore\n",
+        "def_blob_store.upload_files(files=['20news.pkl'], target_path='20newsgroups')"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from azureml.core import Dataset\n",
+        "from azureml.data import OutputFileDatasetConfig\n",
         "\n",
-        "output = PipelineData(\"output\", datastore=def_blob_store)\n",
+        "# create dataset to be used as the input to estimator step\n",
+        "input_data = Dataset.File.from_files(def_blob_store.path('20newsgroups/20news.pkl'))\n",
+        "\n",
+        "# OutputFileDatasetConfig by default write output to the default workspaceblobstore\n",
+        "output = OutputFileDatasetConfig()\n",
         "\n",
         "source_directory = 'estimator_train'"
       ]
@@ -204,10 +212,8 @@
         "\n",
         "- **name:** Name of the step\n",
         "- **estimator:** Estimator object\n",
-        "- **estimator_entry_script_arguments:** \n",
+        "- **estimator_entry_script_arguments:** A list of command-line arguments\n",
         "- **runconfig_pipeline_params:** Override runconfig properties at runtime using key-value pairs each with name of the runconfig property and PipelineParameter for that property\n",
-        "- **inputs:** Inputs\n",
-        "- **outputs:** Output is list of PipelineData\n",
         "- **compute_target:** Compute target to use \n",
         "- **allow_reuse:** Whether the step should reuse previous results when run with the same settings/inputs. If this is false, a new run will always be generated for this step during pipeline execution.\n",
         "- **version:** Optional version tag to denote a change in functionality for the step"
@@ -227,10 +233,8 @@
         "\n",
         "est_step = EstimatorStep(name=\"Estimator_Train\", \n",
         "                         estimator=est, \n",
-        "                         estimator_entry_script_arguments=[\"--datadir\", input_data, \"--output\", output],\n",
+        "                         estimator_entry_script_arguments=[\"--datadir\", input_data.as_mount(), \"--output\", output],\n",
         "                         runconfig_pipeline_params=None, \n",
-        "                         inputs=[input_data], \n",
-        "                         outputs=[output], \n",
         "                         compute_target=cpu_cluster)"
       ]
     },

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-parameter-tuning-with-hyperdrive.ipynb

@@ -42,11 +42,9 @@
       "outputs": [],
       "source": [
         "import azureml.core\n",
-        "from azureml.core import Workspace, Experiment\n",
-        "from azureml.core.datastore import Datastore\n",
+        "from azureml.core import Workspace, Experiment, Datastore, Dataset\n",
         "from azureml.core.compute import ComputeTarget, AmlCompute\n",
         "from azureml.exceptions import ComputeTargetException\n",
-        "from azureml.data.data_reference import DataReference\n",
         "from azureml.pipeline.steps import HyperDriveStep, HyperDriveStepRun\n",
         "from azureml.pipeline.core import Pipeline, PipelineData\n",
         "from azureml.train.dnn import TensorFlow\n",
@@ -179,8 +177,25 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "ds = ws.get_default_datastore()\n",
-        "ds.upload(src_dir='./data/mnist', target_path='mnist', overwrite=True, show_progress=True)"
+        "datastore = ws.get_default_datastore()\n",
+        "datastore.upload(src_dir='./data/mnist', target_path='mnist', overwrite=True, show_progress=True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Create Azure Machine Learning datasets\n",
+        "By creating a dataset, you create a reference to the data source location. If you applied any subsetting transformations to the dataset, they will be stored in the dataset as well. The data remains in its existing location, so no extra storage cost is incurred."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "dataset = Dataset.File.from_files(datastore.path('mnist'))"
       ]
     },
     {
@@ -204,7 +219,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "cluster_name = \"gpu-cluster\"\n",
+        "cluster_name = \"amlcomp\"\n",
         "\n",
         "try:\n",
         "    compute_target = ComputeTarget(workspace=ws, name=cluster_name)\n",
@@ -264,7 +279,8 @@
         "                 compute_target=compute_target,\n",
         "                 entry_script='tf_mnist.py', \n",
         "                 use_gpu=True,\n",
-        "                 framework_version='1.13')"
+        "                 framework_version='2.0',\n",
+        "                 pip_packages=['azureml-dataset-runtime[pandas,fuse]'])"
       ]
     },
     {
@@ -344,7 +360,7 @@
         "## Add HyperDrive as a step of pipeline\n",
         "\n",
         "### Setup an input for the hypderdrive step\n",
-        "Let's setup a data reference for inputs of hyperdrive step."
+        "You can mount dataset to remote compute."
       ]
     },
     {
@@ -353,9 +369,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "data_folder = DataReference(\n",
-        "    datastore=ds,\n",
-        "    data_reference_name=\"mnist_data\")"
+        "data_folder = dataset.as_mount()"
       ]
     },
     {
@@ -386,7 +400,7 @@
       "source": [
         "metrics_output_name = 'metrics_output'\n",
         "metrics_data = PipelineData(name='metrics_data',\n",
-        "                             datastore=ds,\n",
+        "                             datastore=datastore,\n",
         "                             pipeline_output_name=metrics_output_name)\n",
         "\n",
         "hd_step_name='hd_step01'\n",
@@ -577,7 +591,7 @@
       "name": "python",
       "nbconvert_exporter": "python",
       "pygments_lexer": "ipython3",
-      "version": "3.6.7"
+      "version": "3.6.9"
     },
     "order_index": 8,
     "star_tag": [

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-parameter-tuning-with-hyperdrive.yml

@@ -6,3 +6,4 @@ dependencies:
   - matplotlib
   - numpy
   - pandas_ml
+  - azureml-dataset-runtime[pandas,fuse]

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-showcasing-datapath-and-pipelineparameter.ipynb

@@ -87,7 +87,7 @@
       "source": [
         "## Create an Azure ML experiment\n",
         "\n",
-        "Let's create an experiment named \"automl-classification\" and a folder to hold the training scripts. The script runs will be recorded under the experiment in Azure."
+        "Let's create an experiment named \"showcasing-datapath\" and a folder to hold the training scripts. The script runs will be recorded under the experiment in Azure."
       ]
     },
     {
@@ -479,7 +479,7 @@
   "metadata": {
     "authors": [
       {
-        "name": "sanpil"
+        "name": "shbijlan"
       }
     ],
     "category": "tutorial",

how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/tf_mnist.py

@@ -4,34 +4,103 @@
 import numpy as np
 import argparse
 import os
+import re
 import tensorflow as tf
+import time
+import glob
 
 from azureml.core import Run
 from utils import load_data
+from tensorflow.keras import Model, layers
 
-print("TensorFlow version:", tf.VERSION)
+
+# Create TF Model.
+class NeuralNet(Model):
+    # Set layers.
+    def __init__(self):
+        super(NeuralNet, self).__init__()
+        # First hidden layer.
+        self.h1 = layers.Dense(n_h1, activation=tf.nn.relu)
+        # Second hidden layer.
+        self.h2 = layers.Dense(n_h2, activation=tf.nn.relu)
+        self.out = layers.Dense(n_outputs)
+
+    # Set forward pass.
+    def call(self, x, is_training=False):
+        x = self.h1(x)
+        x = self.h2(x)
+        x = self.out(x)
+        if not is_training:
+            # Apply softmax when not training.
+            x = tf.nn.softmax(x)
+        return x
+
+
+def cross_entropy_loss(y, logits):
+    # Convert labels to int 64 for tf cross-entropy function.
+    y = tf.cast(y, tf.int64)
+    # Apply softmax to logits and compute cross-entropy.
+    loss = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=logits)
+    # Average loss across the batch.
+    return tf.reduce_mean(loss)
+
+
+# Accuracy metric.
+def accuracy(y_pred, y_true):
+    # Predicted class is the index of highest score in prediction vector (i.e. argmax).
+    correct_prediction = tf.equal(tf.argmax(y_pred, 1), tf.cast(y_true, tf.int64))
+    return tf.reduce_mean(tf.cast(correct_prediction, tf.float32), axis=-1)
+
+
+# Optimization process.
+def run_optimization(x, y):
+    # Wrap computation inside a GradientTape for automatic differentiation.
+    with tf.GradientTape() as g:
+        # Forward pass.
+        logits = neural_net(x, is_training=True)
+        # Compute loss.
+        loss = cross_entropy_loss(y, logits)
+
+    # Variables to update, i.e. trainable variables.
+    trainable_variables = neural_net.trainable_variables
+
+    # Compute gradients.
+    gradients = g.gradient(loss, trainable_variables)
+
+    # Update W and b following gradients.
+    optimizer.apply_gradients(zip(gradients, trainable_variables))
+
+
+print("TensorFlow version:", tf.__version__)
 
 parser = argparse.ArgumentParser()
-parser.add_argument('--data-folder', type=str, dest='data_folder', help='data folder mounting point')
-parser.add_argument('--batch-size', type=int, dest='batch_size', default=50, help='mini batch size for training')
-parser.add_argument('--first-layer-neurons', type=int, dest='n_hidden_1', default=100,
+parser.add_argument('--data-folder', type=str, dest='data_folder', default='data', help='data folder mounting point')
+parser.add_argument('--batch-size', type=int, dest='batch_size', default=128, help='mini batch size for training')
+parser.add_argument('--first-layer-neurons', type=int, dest='n_hidden_1', default=128,
                     help='# of neurons in the first layer')
-parser.add_argument('--second-layer-neurons', type=int, dest='n_hidden_2', default=100,
+parser.add_argument('--second-layer-neurons', type=int, dest='n_hidden_2', default=128,
                     help='# of neurons in the second layer')
 parser.add_argument('--learning-rate', type=float, dest='learning_rate', default=0.01, help='learning rate')
+parser.add_argument('--resume-from', type=str, default=None,
+                    help='location of the model or checkpoint files from where to resume the training')
 args = parser.parse_args()
 
-data_folder = os.path.join(args.data_folder, 'mnist')
+previous_model_location = args.resume_from
+# You can also use environment variable to get the model/checkpoint files location
+# previous_model_location = os.path.expandvars(os.getenv("AZUREML_DATAREFERENCE_MODEL_LOCATION", None))
 
-print('training dataset is stored here:', data_folder)
+data_folder = args.data_folder
+print('Data folder:', data_folder)
 
+# load train and test set into numpy arrays
+# note we scale the pixel intensity values to 0-1 (by dividing it with 255.0) so the model can converge faster.
 X_train = load_data(os.path.join(data_folder, 'train-images.gz'), False) / 255.0
 X_test = load_data(os.path.join(data_folder, 'test-images.gz'), False) / 255.0
-
 y_train = load_data(os.path.join(data_folder, 'train-labels.gz'), True).reshape(-1)
 y_test = load_data(os.path.join(data_folder, 'test-labels.gz'), True).reshape(-1)
 
 print(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep='\n')
+
 training_set_size = X_train.shape[0]
 
 n_inputs = 28 * 28
@@ -39,68 +108,79 @@ n_h1 = args.n_hidden_1
 n_h2 = args.n_hidden_2
 n_outputs = 10
 learning_rate = args.learning_rate
-n_epochs = 50
+n_epochs = 20
 batch_size = args.batch_size
 
-with tf.name_scope('network'):
-    # construct the DNN
-    X = tf.placeholder(tf.float32, shape=(None, n_inputs), name='X')
-    y = tf.placeholder(tf.int64, shape=(None), name='y')
-    h1 = tf.layers.dense(X, n_h1, activation=tf.nn.relu, name='h1')
-    h2 = tf.layers.dense(h1, n_h2, activation=tf.nn.relu, name='h2')
-    output = tf.layers.dense(h2, n_outputs, name='output')
+# Build neural network model.
+neural_net = NeuralNet()
 
-with tf.name_scope('train'):
-    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y, logits=output)
-    loss = tf.reduce_mean(cross_entropy, name='loss')
-    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
-    train_op = optimizer.minimize(loss)
-
-with tf.name_scope('eval'):
-    correct = tf.nn.in_top_k(output, y, 1)
-    acc_op = tf.reduce_mean(tf.cast(correct, tf.float32))
-
-init = tf.global_variables_initializer()
-saver = tf.train.Saver()
+# Stochastic gradient descent optimizer.
+optimizer = tf.optimizers.SGD(learning_rate)
 
 # start an Azure ML run
 run = Run.get_context()
 
-with tf.Session() as sess:
-    init.run()
-    for epoch in range(n_epochs):
+if previous_model_location:
+    # Restore variables from latest checkpoint.
+    checkpoint = tf.train.Checkpoint(model=neural_net, optimizer=optimizer)
+    checkpoint_file_path = tf.train.latest_checkpoint(previous_model_location)
+    checkpoint.restore(checkpoint_file_path)
+    checkpoint_filename = os.path.basename(checkpoint_file_path)
+    num_found = re.search(r'\d+', checkpoint_filename)
+    if num_found:
+        start_epoch = int(num_found.group(0))
+        print("Resuming from epoch {}".format(str(start_epoch)))
 
-        # randomly shuffle training set
-        indices = np.random.permutation(training_set_size)
-        X_train = X_train[indices]
-        y_train = y_train[indices]
+start_time = time.perf_counter()
+for epoch in range(0, n_epochs):
 
-        # batch index
-        b_start = 0
-        b_end = b_start + batch_size
-        for _ in range(training_set_size // batch_size):
-            # get a batch
-            X_batch, y_batch = X_train[b_start: b_end], y_train[b_start: b_end]
+    # randomly shuffle training set
+    indices = np.random.permutation(training_set_size)
+    X_train = X_train[indices]
+    y_train = y_train[indices]
 
-            # update batch index for the next batch
-            b_start = b_start + batch_size
-            b_end = min(b_start + batch_size, training_set_size)
+    # batch index
+    b_start = 0
+    b_end = b_start + batch_size
+    for _ in range(training_set_size // batch_size):
+        # get a batch
+        X_batch, y_batch = X_train[b_start: b_end], y_train[b_start: b_end]
 
-            # train
-            sess.run(train_op, feed_dict={X: X_batch, y: y_batch})
-        # evaluate training set
-        acc_train = acc_op.eval(feed_dict={X: X_batch, y: y_batch})
-        # evaluate validation set
-        acc_val = acc_op.eval(feed_dict={X: X_test, y: y_test})
+        # update batch index for the next batch
+        b_start = b_start + batch_size
+        b_end = min(b_start + batch_size, training_set_size)
 
-        # log accuracies
-        run.log('training_acc', np.float(acc_train))
-        run.log('validation_acc', np.float(acc_val))
-        print(epoch, '-- Training accuracy:', acc_train, '\b Validation accuracy:', acc_val)
-        y_hat = np.argmax(output.eval(feed_dict={X: X_test}), axis=1)
+        # train
+        run_optimization(X_batch, y_batch)
 
-    run.log('final_acc', np.float(acc_val))
+    # evaluate training set
+    pred = neural_net(X_batch, is_training=False)
+    acc_train = accuracy(pred, y_batch)
 
-    os.makedirs('./outputs/model', exist_ok=True)
-    # files saved in the "./outputs" folder are automatically uploaded into run history
-    saver.save(sess, './outputs/model/mnist-tf.model')
+    # evaluate validation set
+    pred = neural_net(X_test, is_training=False)
+    acc_val = accuracy(pred, y_test)
+
+    # log accuracies
+    run.log('training_acc', np.float(acc_train))
+    run.log('validation_acc', np.float(acc_val))
+    print(epoch, '-- Training accuracy:', acc_train, '\b Validation accuracy:', acc_val)
+
+    # Save checkpoints in the "./outputs" folder so that they are automatically uploaded into run history.
+    checkpoint_dir = './outputs/'
+    checkpoint = tf.train.Checkpoint(model=neural_net, optimizer=optimizer)
+
+    if epoch % 2 == 0:
+        checkpoint.save(checkpoint_dir)
+
+run.log('final_acc', np.float(acc_val))
+os.makedirs('./outputs/model', exist_ok=True)
+
+# files saved in the "./outputs" folder are automatically uploaded into run history
+# this is workaround for https://github.com/tensorflow/tensorflow/issues/33913 and will be fixed once we move to >tf2.1
+neural_net._set_inputs(X_train)
+tf.saved_model.save(neural_net, './outputs/model/')
+
+stop_time = time.perf_counter()
+training_time = (stop_time - start_time) * 1000
+print("Total time in milliseconds for training: {}".format(str(training_time)))

how-to-use-azureml/machine-learning-pipelines/nyc-taxi-data-regression-model-building/scripts/prepdata/transform.py

@@ -29,8 +29,8 @@ print("Argument 2(output final transformed taxi data): %s" % args.output_transfo
 # use the drop_columns() function to delete the original fields as the newly generated features are preferred.
 # Rename the rest of the fields to use meaningful descriptions.
 
-normalized_df = normalized_df.astype({"pickup_date": 'datetime64', "dropoff_date": 'datetime64',
-                                      "pickup_time": 'datetime64', "dropoff_time": 'datetime64',
+normalized_df = normalized_df.astype({"pickup_date": 'datetime64[ns]', "dropoff_date": 'datetime64[ns]',
+                                      "pickup_time": 'datetime64[us]', "dropoff_time": 'datetime64[us]',
                                       "distance": 'float64', "cost": 'float64'})
 
 normalized_df["pickup_weekday"] = normalized_df["pickup_date"].dt.dayofweek

how-to-use-azureml/ml-frameworks/scikit-learn/training/train-hyperparameter-tune-deploy-with-sklearn/train-hyperparameter-tune-deploy-with-sklearn.ipynb

@@ -418,7 +418,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "from azureml.train.hyperdrive.runconfig import HyperDriveRunConfig\n",
+        "from azureml.train.hyperdrive.runconfig import HyperDriveConfig\n",
         "from azureml.train.hyperdrive.sampling import RandomParameterSampling\n",
         "from azureml.train.hyperdrive.run import PrimaryMetricGoal\n",
         "from azureml.train.hyperdrive.parameter_expressions import choice\n",
@@ -430,12 +430,12 @@
         "    }\n",
         ")\n",
         "\n",
-        "hyperdrive_run_config = HyperDriveRunConfig(estimator=estimator,\n",
-        "                                            hyperparameter_sampling=param_sampling, \n",
-        "                                            primary_metric_name='Accuracy',\n",
-        "                                            primary_metric_goal=PrimaryMetricGoal.MAXIMIZE,\n",
-        "                                            max_total_runs=12,\n",
-        "                                            max_concurrent_runs=4)"
+        "hyperdrive_config = HyperDriveConfig(estimator=estimator,\n",
+        "                                     hyperparameter_sampling=param_sampling, \n",
+        "                                     primary_metric_name='Accuracy',\n",
+        "                                     primary_metric_goal=PrimaryMetricGoal.MAXIMIZE,\n",
+        "                                     max_total_runs=12,\n",
+        "                                     max_concurrent_runs=4)"
       ]
     },
     {
@@ -452,7 +452,7 @@
       "outputs": [],
       "source": [
         "# start the HyperDrive run\n",
-        "hyperdrive_run = experiment.submit(hyperdrive_run_config)"
+        "hyperdrive_run = experiment.submit(hyperdrive_config)"
       ]
     },
     {

how-to-use-azureml/ml-frameworks/tensorflow/training/distributed-tensorflow-with-horovod/distributed-tensorflow-with-horovod.ipynb

@@ -158,7 +158,7 @@
       "source": [
         "from azureml.core import Dataset\n",
         "\n",
-        "web_paths = ['http://mattmahoney.net/dc/text8.zip']\n",
+        "web_paths = ['https://azureopendatastorage.blob.core.windows.net/testpublic/text8.zip']\n",
         "dataset = Dataset.File.from_files(path=web_paths)"
       ]
     },

how-to-use-azureml/ml-frameworks/using-mlflow/train-and-deploy-keras-auto-logging/train-and-deploy-keras-auto-logging.ipynb

@@ -13,7 +13,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/using-mlflow/train-and-deploy-keras-auto-logging/train-and-deploy-keras-auto-logging.png)"
+        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/ml-frameworks/using-mlflow/train-and-deploy-keras-auto-logging/train-and-deploy-keras-auto-logging.png)"
       ]
     },
     {

how-to-use-azureml/ml-frameworks/using-mlflow/train-and-deploy-pytorch/train-and-deploy-pytorch.ipynb

@@ -13,7 +13,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/using-mlflow/train-and-deploy-pytorch/train-and-deploy-pytorch.png)"
+        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/ml-frameworks/using-mlflow/train-and-deploy-pytorch/train-and-deploy-pytorch.png)"
       ]
     },
     {

how-to-use-azureml/monitor-models/data-drift/config.py

@@ -1,31 +0,0 @@
-# imports
-import pickle
-from datetime import datetime
-from azureml.opendatasets import NoaaIsdWeather
-from sklearn.linear_model import LinearRegression
-
-# get weather dataset
-start = datetime(2019, 1, 1)
-end = datetime(2019, 1, 14)
-isd = NoaaIsdWeather(start, end)
-
-# convert to pandas dataframe and filter down
-df = isd.to_pandas_dataframe().fillna(0)
-df = df[df['stationName'].str.contains('FLORIDA', regex=True, na=False)]
-
-# features for training
-X_features = ['latitude', 'longitude', 'temperature', 'windAngle', 'windSpeed']
-y_features = ['elevation']
-
-# write the training dataset to csv
-training_dataset = df[X_features + y_features]
-training_dataset.to_csv('training.csv', index=False)
-
-# train the model
-X = training_dataset[X_features]
-y = training_dataset[y_features]
-model = LinearRegression().fit(X, y)
-
-# save the model as a .pkl file
-with open('elevation-regression-model.pkl', 'wb') as f:
-    pickle.dump(model, f)

how-to-use-azureml/monitor-models/data-drift/dataset/testing.csv

@@ -1,346 +0,0 @@
-latitude,longitude,temperature,windAngle,windSpeed,elevation
-26.536,-81.755,17.8,10.0,2.1,9.0
-26.536,-81.755,16.7,360.0,1.5,9.0
-26.536,-81.755,16.1,350.0,1.5,9.0
-26.536,-81.755,15.0,0.0,0.0,9.0
-26.536,-81.755,14.4,350.0,1.5,9.0
-26.536,-81.755,0.0,0.0,0.0,9.0
-26.536,-81.755,13.9,360.0,2.1,9.0
-26.536,-81.755,13.3,350.0,1.5,9.0
-26.536,-81.755,13.3,10.0,2.1,9.0
-26.536,-81.755,13.3,360.0,1.5,9.0
-26.536,-81.755,13.3,0.0,0.0,9.0
-26.536,-81.755,12.2,0.0,0.0,9.0
-26.536,-81.755,11.7,0.0,0.0,9.0
-26.536,-81.755,14.4,0.0,0.0,9.0
-26.536,-81.755,17.2,10.0,2.6,9.0
-26.536,-81.755,20.0,20.0,2.6,9.0
-26.536,-81.755,22.2,10.0,3.6,9.0
-26.536,-81.755,23.3,30.0,4.6,9.0
-26.536,-81.755,23.3,330.0,2.6,9.0
-26.536,-81.755,24.4,0.0,0.0,9.0
-26.536,-81.755,25.0,360.0,3.1,9.0
-26.536,-81.755,24.4,20.0,4.1,9.0
-26.536,-81.755,23.3,10.0,2.6,9.0
-26.536,-81.755,21.1,30.0,2.1,9.0
-26.536,-81.755,18.3,0.0,0.0,9.0
-26.536,-81.755,17.2,30.0,2.1,9.0
-26.536,-81.755,15.6,60.0,2.6,9.0
-26.536,-81.755,15.6,0.0,0.0,9.0
-26.536,-81.755,13.9,60.0,2.6,9.0
-26.536,-81.755,12.8,70.0,2.6,9.0
-26.536,-81.755,0.0,0.0,0.0,9.0
-26.536,-81.755,11.7,70.0,2.1,9.0
-26.536,-81.755,12.2,20.0,2.1,9.0
-26.536,-81.755,11.7,30.0,1.5,9.0
-26.536,-81.755,11.1,40.0,2.1,9.0
-26.536,-81.755,12.2,40.0,2.6,9.0
-26.536,-81.755,12.2,30.0,2.6,9.0
-26.536,-81.755,12.2,0.0,0.0,9.0
-26.536,-81.755,15.0,30.0,6.2,9.0
-26.536,-81.755,17.2,50.0,3.6,9.0
-26.536,-81.755,20.6,60.0,5.1,9.0
-26.536,-81.755,22.8,50.0,4.6,9.0
-26.536,-81.755,24.4,80.0,6.2,9.0
-26.536,-81.755,25.0,100.0,5.7,9.0
-26.536,-81.755,25.6,60.0,3.1,9.0
-26.536,-81.755,25.6,80.0,4.6,9.0
-26.536,-81.755,25.0,90.0,5.1,9.0
-26.536,-81.755,24.4,80.0,5.1,9.0
-26.536,-81.755,21.1,60.0,2.6,9.0
-26.536,-81.755,19.4,70.0,3.6,9.0
-26.536,-81.755,18.3,70.0,2.6,9.0
-26.536,-81.755,18.3,80.0,2.6,9.0
-26.536,-81.755,17.2,60.0,1.5,9.0
-26.536,-81.755,16.1,70.0,2.6,9.0
-26.536,-81.755,15.6,70.0,2.6,9.0
-26.536,-81.755,0.0,0.0,0.0,9.0
-26.536,-81.755,16.1,50.0,2.6,9.0
-26.536,-81.755,15.6,50.0,2.1,9.0
-26.536,-81.755,15.0,50.0,1.5,9.0
-26.536,-81.755,15.0,0.0,0.0,9.0
-26.536,-81.755,15.0,0.0,0.0,9.0
-26.536,-81.755,14.4,0.0,0.0,9.0
-26.536,-81.755,14.4,30.0,4.1,9.0
-26.536,-81.755,16.1,40.0,1.5,9.0
-26.536,-81.755,19.4,0.0,1.5,9.0
-26.536,-81.755,22.8,90.0,2.6,9.0
-26.536,-81.755,24.4,130.0,3.6,9.0
-26.536,-81.755,25.6,100.0,4.6,9.0
-26.536,-81.755,26.1,120.0,3.1,9.0
-26.536,-81.755,26.7,0.0,2.6,9.0
-26.536,-81.755,27.2,0.0,0.0,9.0
-26.536,-81.755,27.2,40.0,3.1,9.0
-26.536,-81.755,26.1,30.0,1.5,9.0
-26.536,-81.755,22.8,310.0,2.1,9.0
-26.536,-81.755,23.3,330.0,2.1,9.0
--34.067,-56.238,17.5,30.0,3.1,68.0
--34.067,-56.238,21.2,30.0,5.7,68.0
--34.067,-56.238,24.5,30.0,3.1,68.0
--34.067,-56.238,27.5,330.0,3.6,68.0
--34.067,-56.238,29.2,30.0,4.1,68.0
--34.067,-56.238,31.0,20.0,4.6,68.0
--34.067,-56.238,33.0,360.0,2.6,68.0
--34.067,-56.238,33.6,60.0,3.1,68.0
--34.067,-56.238,33.6,30.0,3.6,68.0
--34.067,-56.238,18.6,40.0,3.1,68.0
--34.067,-56.238,22.0,120.0,1.5,68.0
--34.067,-56.238,25.0,120.0,2.6,68.0
--34.067,-56.238,28.6,50.0,3.1,68.0
--34.067,-56.238,30.6,50.0,4.1,68.0
--34.067,-56.238,31.5,30.0,6.7,68.0
--34.067,-56.238,32.0,40.0,7.2,68.0
--34.067,-56.238,33.0,30.0,5.7,68.0
--34.067,-56.238,33.2,360.0,3.6,68.0
--34.067,-56.238,20.6,30.0,3.1,68.0
--34.067,-56.238,21.2,0.0,0.0,68.0
--34.067,-56.238,22.0,210.0,3.1,68.0
--34.067,-56.238,23.0,210.0,3.6,68.0
--34.067,-56.238,24.0,180.0,6.7,68.0
--34.067,-56.238,24.5,210.0,7.2,68.0
--34.067,-56.238,21.0,180.0,8.2,68.0
--34.067,-56.238,20.0,180.0,6.7,68.0
--34.083,-56.233,20.2,180.0,7.2,68.0
--29.917,-71.2,16.6,290.0,4.1,146.0
--29.916,-71.2,17.0,290.0,4.1,147.0
--29.916,-71.2,16.0,310.0,3.1,147.0
--29.916,-71.2,16.0,300.0,2.1,147.0
--29.917,-71.2,15.1,0.0,0.0,146.0
--29.916,-71.2,15.0,0.0,1.0,147.0
--29.916,-71.2,15.0,160.0,1.0,147.0
--29.916,-71.2,15.0,120.0,1.0,147.0
--29.917,-71.2,14.3,190.0,1.0,146.0
--29.916,-71.2,14.0,190.0,1.0,147.0
--29.916,-71.2,14.0,0.0,0.0,147.0
--29.916,-71.2,14.0,100.0,3.1,147.0
--29.917,-71.2,12.9,0.0,0.0,146.0
--29.916,-71.2,13.0,0.0,1.0,147.0
--29.916,-71.2,14.0,0.0,0.5,147.0
--29.916,-71.2,15.0,0.0,0.5,147.0
--29.917,-71.2,15.9,0.0,0.0,146.0
--29.916,-71.2,16.0,0.0,0.0,147.0
--29.916,-71.2,17.0,270.0,4.6,147.0
--29.916,-71.2,19.0,260.0,4.1,147.0
--29.917,-71.2,18.1,270.0,6.2,146.0
--29.916,-71.2,18.0,270.0,6.2,147.0
--29.916,-71.2,19.0,270.0,6.2,147.0
--29.916,-71.2,20.0,260.0,5.1,147.0
--29.917,-71.2,19.6,280.0,6.2,146.0
--29.916,-71.2,20.0,280.0,6.2,147.0
--29.916,-71.2,20.0,270.0,6.2,147.0
--29.916,-71.2,19.0,280.0,6.7,147.0
--29.917,-71.2,18.3,270.0,5.7,146.0
--29.916,-71.2,18.0,270.0,5.7,147.0
--29.916,-71.2,18.0,0.0,0.0,147.0
--29.916,-71.2,17.0,280.0,4.6,147.0
--29.917,-71.2,15.9,280.0,4.1,146.0
--29.916,-71.2,16.0,280.0,4.1,147.0
--29.916,-71.2,15.0,280.0,3.6,147.0
--29.916,-71.2,15.0,280.0,3.6,147.0
--29.917,-71.2,15.4,280.0,4.1,146.0
--29.916,-71.2,15.0,280.0,4.1,147.0
--29.916,-71.2,16.0,240.0,2.1,147.0
--29.916,-71.2,15.0,0.0,0.5,147.0
--29.917,-71.2,15.8,80.0,3.6,146.0
--29.916,-71.2,16.0,80.0,3.6,147.0
--29.916,-71.2,16.0,10.0,1.5,147.0
--29.916,-71.2,16.0,100.0,1.5,147.0
--29.917,-71.2,15.3,130.0,1.5,146.0
--29.916,-71.2,15.0,130.0,1.5,147.0
--29.916,-71.2,15.0,110.0,1.0,147.0
--29.916,-71.2,16.0,280.0,6.2,147.0
--29.917,-71.2,15.9,240.0,3.6,146.0
--29.916,-71.2,16.0,240.0,3.6,147.0
--29.916,-71.2,16.0,240.0,3.1,147.0
--29.916,-71.2,16.0,220.0,3.1,147.0
--29.917,-71.2,16.4,260.0,3.1,146.0
--29.916,-71.2,16.0,260.0,3.1,147.0
--29.916,-71.2,17.0,230.0,2.6,147.0
--29.916,-71.2,18.0,0.0,1.5,147.0
--29.917,-71.2,20.3,340.0,2.6,146.0
--29.916,-71.2,20.0,340.0,2.6,147.0
--29.916,-71.2,21.0,270.0,5.1,147.0
--29.916,-71.2,20.0,270.0,6.7,147.0
--29.917,-71.2,19.2,280.0,6.7,146.0
--29.916,-71.2,19.0,280.0,6.7,147.0
--29.916,-71.2,19.0,310.0,2.6,147.0
--29.916,-71.2,18.0,270.0,5.1,147.0
--29.917,-71.2,17.0,300.0,4.6,146.0
--29.916,-71.2,17.0,300.0,4.6,147.0
--29.916,-71.2,17.0,300.0,3.6,147.0
--29.916,-71.2,17.0,290.0,3.1,147.0
--29.917,-71.2,16.3,290.0,2.1,146.0
--29.916,-71.2,16.0,290.0,2.1,147.0
--29.916,-71.2,17.0,270.0,1.0,147.0
--29.916,-71.2,17.0,0.0,0.5,147.0
--29.917,-71.2,16.5,160.0,2.1,146.0
--29.916,-71.2,17.0,160.0,2.1,147.0
--29.916,-71.2,15.0,120.0,3.1,147.0
--29.916,-71.2,16.0,180.0,1.5,147.0
--29.917,-71.2,14.7,0.0,0.0,146.0
--29.916,-71.2,15.0,0.0,1.0,147.0
--29.916,-71.2,15.0,300.0,1.0,147.0
--29.916,-71.2,16.0,0.0,0.0,147.0
--29.917,-71.2,18.5,110.0,1.0,146.0
--29.916,-71.2,19.0,110.0,1.0,147.0
--29.916,-71.2,20.0,270.0,3.6,147.0
--29.916,-71.2,20.0,270.0,5.7,147.0
--29.917,-71.2,20.0,280.0,6.2,146.0
--29.916,-71.2,20.0,280.0,6.2,147.0
--29.916,-71.2,21.0,290.0,6.7,147.0
--29.916,-71.2,20.0,270.0,6.2,147.0
--29.917,-71.2,21.0,260.0,6.7,146.0
--29.916,-71.2,21.0,260.0,6.7,147.0
--29.916,-71.2,20.0,270.0,6.2,147.0
--29.916,-71.2,19.0,260.0,5.1,147.0
--29.916,-71.2,18.0,280.0,4.6,147.0
--29.917,-71.2,17.5,280.0,3.1,146.0
--29.916,-71.2,18.0,280.0,3.1,147.0
-30.349,-85.788,11.1,0.0,0.0,21.0
-30.349,-85.788,11.1,0.0,0.0,21.0
-30.349,-85.788,9.4,0.0,0.0,21.0
-30.349,-85.788,9.4,0.0,0.0,21.0
-30.349,-85.788,8.3,300.0,2.1,21.0
-30.349,-85.788,11.1,280.0,1.5,21.0
-30.349,-85.788,0.0,0.0,0.0,21.0
-30.349,-85.788,10.6,320.0,3.1,21.0
-30.349,-85.788,9.4,310.0,3.1,21.0
-30.349,-85.788,7.8,320.0,2.6,21.0
-30.349,-85.788,6.1,340.0,2.1,21.0
-30.349,-85.788,6.7,330.0,2.6,21.0
-30.349,-85.788,6.1,310.0,1.5,21.0
-30.349,-85.788,7.2,310.0,2.1,21.0
-30.349,-85.788,12.8,360.0,3.1,21.0
-30.349,-85.788,15.0,0.0,3.1,21.0
-30.349,-85.788,16.7,20.0,4.6,21.0
-30.349,-85.788,18.9,30.0,5.1,21.0
-30.349,-85.788,19.4,10.0,4.1,21.0
-30.349,-85.788,21.1,330.0,2.6,21.0
-30.349,-85.788,21.1,10.0,4.6,21.0
-30.349,-85.788,21.7,360.0,4.1,21.0
-30.349,-85.788,21.7,30.0,2.1,21.0
-30.349,-85.788,21.7,330.0,2.6,21.0
-30.349,-85.788,16.1,350.0,2.1,21.0
-30.349,-85.788,11.7,0.0,0.0,21.0
-30.349,-85.788,8.9,0.0,0.0,21.0
-30.349,-85.788,9.4,0.0,0.0,21.0
-30.349,-85.788,7.8,0.0,0.0,21.0
-30.349,-85.788,11.1,30.0,3.1,21.0
-30.349,-85.788,7.2,0.0,0.0,21.0
-30.349,-85.788,7.2,0.0,0.0,21.0
-30.349,-85.788,0.0,0.0,0.0,21.0
-30.349,-85.788,7.8,30.0,2.1,21.0
-30.349,-85.788,8.3,40.0,2.6,21.0
-30.349,-85.788,7.2,50.0,1.5,21.0
-30.349,-85.788,8.3,60.0,1.5,21.0
-30.349,-85.788,5.6,40.0,2.1,21.0
-30.349,-85.788,6.7,40.0,2.1,21.0
-30.349,-85.788,7.8,50.0,3.1,21.0
-30.349,-85.788,11.7,70.0,2.6,21.0
-30.349,-85.788,15.6,70.0,3.1,21.0
-30.349,-85.788,18.9,100.0,3.6,21.0
-30.349,-85.788,20.0,130.0,3.6,21.0
-30.349,-85.788,21.1,140.0,4.1,21.0
-30.349,-85.788,21.7,150.0,4.1,21.0
-30.349,-85.788,21.7,170.0,3.1,21.0
-30.349,-85.788,22.2,170.0,3.1,21.0
-30.349,-85.788,20.6,0.0,0.0,21.0
-30.349,-85.788,17.2,0.0,0.0,21.0
-30.349,-85.788,14.4,0.0,0.0,21.0
-30.349,-85.788,12.8,100.0,1.5,21.0
-30.349,-85.788,13.3,100.0,1.5,21.0
-30.349,-85.788,10.6,0.0,0.0,21.0
-30.349,-85.788,9.4,0.0,0.0,21.0
-30.349,-85.788,7.8,0.0,0.0,21.0
-30.358,-85.799,8.3,0.0,0.0,21.0
-30.349,-85.788,0.0,0.0,0.0,21.0
-30.358,-85.799,6.7,0.0,0.0,21.0
-30.358,-85.799,7.2,0.0,0.0,21.0
-30.358,-85.799,7.2,0.0,0.0,21.0
-30.358,-85.799,8.3,50.0,1.5,21.0
-30.358,-85.799,9.4,0.0,0.0,21.0
-30.358,-85.799,8.9,0.0,0.0,21.0
-30.358,-85.799,10.0,340.0,1.5,21.0
-30.358,-85.799,12.8,40.0,1.5,21.0
-30.358,-85.799,16.7,100.0,2.1,21.0
-30.358,-85.799,21.1,100.0,1.5,21.0
-30.358,-85.799,23.3,0.0,0.0,21.0
-30.358,-85.799,25.0,180.0,4.6,21.0
-30.358,-85.799,24.4,230.0,3.6,21.0
-30.358,-85.799,25.0,210.0,4.1,21.0
-30.358,-85.799,23.9,170.0,4.1,21.0
-30.358,-85.799,22.8,0.0,0.0,21.0
-30.358,-85.799,19.4,0.0,0.0,21.0
-30.358,-85.799,17.8,140.0,2.1,21.0
-60.383,5.333,-0.7,0.0,0.0,36.0
-60.383,5.333,0.6,270.0,2.0,36.0
-60.383,5.333,-0.9,120.0,1.0,36.0
-60.383,5.333,-1.6,130.0,2.0,36.0
-60.383,5.333,-1.4,150.0,1.0,36.0
-60.383,5.333,-1.7,0.0,0.0,36.0
-60.383,5.333,-1.7,140.0,1.0,36.0
-60.383,5.333,-1.4,0.0,0.0,36.0
-60.383,5.333,-1.0,0.0,0.0,36.0
-60.383,5.333,-1.0,150.0,1.0,36.0
-60.383,5.333,-0.7,140.0,1.0,36.0
-60.383,5.333,0.5,150.0,1.0,36.0
-60.383,5.333,1.9,0.0,0.0,36.0
-60.383,5.333,1.7,0.0,0.0,36.0
-60.383,5.333,2.1,310.0,2.0,36.0
-60.383,5.333,1.5,90.0,1.0,36.0
-60.383,5.333,1.9,290.0,1.0,36.0
-60.383,5.333,2.0,320.0,1.0,36.0
-60.383,5.333,1.9,330.0,1.0,36.0
-60.383,5.333,1.3,350.0,1.0,36.0
-60.383,5.333,1.5,120.0,1.0,36.0
-60.383,5.333,1.3,150.0,2.0,36.0
-60.383,5.333,0.8,140.0,1.0,36.0
-60.383,5.333,0.3,300.0,1.0,36.0
-60.383,5.333,0.2,140.0,1.0,36.0
-60.383,5.333,0.4,140.0,1.0,36.0
-60.383,5.333,0.5,320.0,1.0,36.0
-60.383,5.333,1.5,330.0,1.0,36.0
-60.383,5.333,1.8,40.0,1.0,36.0
-60.383,5.333,2.3,170.0,1.0,36.0
-60.383,5.333,2.7,140.0,1.0,36.0
-60.383,5.333,3.1,330.0,1.0,36.0
-60.383,5.333,3.8,350.0,1.0,36.0
-60.383,5.333,3.8,140.0,1.0,36.0
-60.383,5.333,4.1,150.0,1.0,36.0
-60.383,5.333,4.4,180.0,1.0,36.0
-60.383,5.333,4.9,300.0,1.0,36.0
-60.383,5.333,5.2,320.0,1.0,36.0
-60.383,5.333,6.7,340.0,1.0,36.0
-60.383,5.333,6.9,250.0,1.0,36.0
-60.383,5.333,7.9,300.0,2.0,36.0
-60.383,5.333,5.5,140.0,1.0,36.0
-60.383,5.333,7.1,140.0,2.0,36.0
-60.383,5.333,7.0,280.0,2.0,36.0
-60.383,5.333,4.6,170.0,1.0,36.0
-60.383,5.333,4.8,330.0,1.0,36.0
-60.383,5.333,6.4,260.0,2.0,36.0
-60.383,5.333,6.2,340.0,1.0,36.0
-60.383,5.333,5.7,320.0,2.0,36.0
-60.383,5.333,5.2,100.0,1.0,36.0
-60.383,5.333,5.1,310.0,1.0,36.0
-60.383,5.333,4.9,290.0,2.0,36.0
-60.383,5.333,4.9,310.0,2.0,36.0
-60.383,5.333,6.1,320.0,2.0,36.0
-60.383,5.333,7.0,250.0,1.0,36.0
-60.383,5.333,5.3,140.0,1.0,36.0
-60.383,5.333,6.9,350.0,1.0,36.0
-60.383,5.333,9.7,110.0,3.0,36.0
-60.383,5.333,10.3,300.0,3.0,36.0
-60.383,5.333,8.7,310.0,1.0,36.0
-60.383,5.333,9.0,270.0,3.0,36.0
-60.383,5.333,11.6,80.0,3.0,36.0
-60.383,5.333,11.4,80.0,4.0,36.0
-60.383,5.333,9.7,70.0,5.0,36.0
-60.383,5.333,9.5,80.0,6.0,36.0
-60.383,5.333,8.7,80.0,5.0,36.0
-60.383,5.333,7.7,80.0,5.0,36.0
-60.383,5.333,8.2,80.0,4.0,36.0
-60.383,5.333,7.7,30.0,1.0,36.0
-60.383,5.333,7.2,310.0,1.0,36.0
-60.383,5.333,6.8,300.0,2.0,36.0
-60.383,5.333,6.7,140.0,1.0,36.0

how-to-use-azureml/monitor-models/data-drift/drift-on-aks.ipynb

@@ -1,578 +0,0 @@
-{
-  "cells": [
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
-        "\n",
-        "Licensed under the MIT License."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/monitor-models/data-drift/drift-on-aks.png)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "# Monitor data drift on models deployed to Azure Kubernetes Service \n",
-        "\n",
-        "In this tutorial, you will setup a data drift monitor on a toy model that predicts elevation based on a few weather factors which will send email alerts if drift is detected."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Prerequisites\n",
-        "If you are using an Azure Machine Learning Compute instance, you are all set. Otherwise, go through the [configuration notebook](../../../configuration.ipynb) first if you haven't already established your connection to the AzureML Workspace."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# Check core SDK version number\n",
-        "import azureml.core\n",
-        "\n",
-        "print('SDK version:', azureml.core.VERSION)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Initialize Workspace\n",
-        "\n",
-        "Initialize a workspace object from persisted configuration."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core import Workspace\n",
-        "\n",
-        "ws = Workspace.from_config()\n",
-        "ws"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Setup training dataset and model\n",
-        "\n",
-        "Setup the training dataset and model in preparation for deployment to the Azure Kubernetes Service. \n",
-        "\n",
-        "The next few cells will:\n",
-        "  * get the default datastore and upload the `training.csv` dataset to the datastore\n",
-        "  * create and register the dataset \n",
-        "  * register the model with the dataset\n",
-        "  \n",
-        "See the `config.py` script in this folder for details on how `training.csv` and `elevation-regression-model.pkl` are created. If you train your model in Azure ML using a Dataset, it will be automatically captured when registering the model from the run. "
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# use default datastore\n",
-        "dstore = ws.get_default_datastore()\n",
-        "\n",
-        "# upload weather data\n",
-        "dstore.upload('dataset', 'drift-on-aks-data', overwrite=True, show_progress=False)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core import Dataset\n",
-        "\n",
-        "# create dataset \n",
-        "dset = Dataset.Tabular.from_delimited_files(dstore.path('drift-on-aks-data/training.csv'))\n",
-        "# register dataset\n",
-        "dset = dset.register(ws, 'drift-demo-dataset')\n",
-        "# get the dataset by name from the workspace\n",
-        "dset = Dataset.get_by_name(ws, 'drift-demo-dataset')"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.model import Model\n",
-        "\n",
-        "# register the model\n",
-        "model = Model.register(model_path='elevation-regression-model.pkl',\n",
-        "                       model_name='elevation-regression-model.pkl',\n",
-        "                       tags={'area': \"weather\", 'type': \"linear regression\"},\n",
-        "                       description='Linear regression model to predict elevation based on the weather',\n",
-        "                       workspace=ws,\n",
-        "                       datasets=[(Dataset.Scenario.TRAINING, dset)]) # need to register the dataset with the model"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Create the inference config\n",
-        "\n",
-        "Create the environment and inference config from the `myenv.yml` and `score.py` files. Notice the [Model Data Collector](https://docs.microsoft.com/azure/machine-learning/service/how-to-enable-data-collection) code included in the scoring script. This dependency is currently required to collect model data, but will be removed in the near future as data collection in Azure Machine Learning webservice endpoints is automated."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core import Environment\n",
-        "\n",
-        "# create the environment from the yml file \n",
-        "env = Environment.from_conda_specification(name='deploytocloudenv', file_path='myenv.yml')"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.model import InferenceConfig\n",
-        "\n",
-        "# create an inference config, combining the environment and entry script \n",
-        "inference_config = InferenceConfig(entry_script='score.py', environment=env)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Create the AKS compute target\n",
-        "\n",
-        "Create an Azure Kubernetes Service compute target to deploy the model to. "
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.compute import AksCompute, ComputeTarget\n",
-        "\n",
-        "# Use the default configuration (you can also provide parameters to customize this).\n",
-        "# For example, to create a dev/test cluster, use:\n",
-        "# prov_config = AksCompute.provisioning_configuration(cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST)\n",
-        "prov_config = AksCompute.provisioning_configuration()\n",
-        "\n",
-        "aks_name = 'drift-aks'\n",
-        "aks_target = ws.compute_targets.get(aks_name)\n",
-        "\n",
-        "# Create the cluster\n",
-        "if not aks_target:\n",
-        "    aks_target = ComputeTarget.create(workspace = ws,\n",
-        "                                      name = aks_name,\n",
-        "                                      provisioning_configuration = prov_config)\n",
-        "\n",
-        "    # Wait for the create process to complete\n",
-        "    aks_target.wait_for_completion(show_output = True)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Deploy the model to AKS \n",
-        "\n",
-        "Deploy the model as a webservice endpoint. Be sure to enable the `collect_model_data` flag so that serving data is collected in blob storage for use by the data drift capability."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.webservice import AksWebservice\n",
-        "\n",
-        "deployment_config = AksWebservice.deploy_configuration(cpu_cores=1, memory_gb=1, collect_model_data=True)\n",
-        "service_name = 'drift-aks-service'\n",
-        "\n",
-        "service = Model.deploy(ws, service_name, [model], inference_config, deployment_config, aks_target)\n",
-        "\n",
-        "service.wait_for_deployment(True)\n",
-        "print(service.state)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Run recent weather data through the webservice \n",
-        "\n",
-        "The below cells take the weather data of Florida from 2019-11-20 to 2019-11-12, filter and transform using the same processes as the training dataset, and runs the data through the service."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# create dataset \n",
-        "tset = Dataset.Tabular.from_delimited_files(dstore.path('drift-on-aks-data/testing.csv'))\n",
-        "\n",
-        "df = tset.to_pandas_dataframe().fillna(0)\n",
-        "\n",
-        "X_features = ['latitude', 'longitude', 'temperature', 'windAngle', 'windSpeed']\n",
-        "y_features = ['elevation']\n",
-        "\n",
-        "X = df[X_features]\n",
-        "y = df[y_features]"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "import json\n",
-        "\n",
-        "data = json.dumps({'data': X.values.tolist()})\n",
-        "\n",
-        "data_encoded = bytes(data, encoding='utf8')\n",
-        "prediction = service.run(input_data=data_encoded)\n",
-        "print(prediction)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Wait 15 minutes for scoring data to be uploaded\n",
-        "\n",
-        "From the Model Data Collector, it can take up to (but usually less than) 15 minutes for data to arrive in your blob storage account. \n",
-        "\n",
-        "Wait 15 minutes to ensure cells below will run."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "import time\n",
-        "\n",
-        "time.sleep(900)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Get scoring dataset thats been created\n",
-        "\n",
-        "Scoring dataset will be created automatically for each model/version/service that has been deployed and registered with name in the format of inference-data-elevation-{model_name}-{model_version}-{service_name}\n",
-        "\n",
-        "Wait 15 minutes to ensure cells below will run."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "scoring_dataset_name = \"inference-data-{0}-{1}-{2}\".format(model.name, model.version, service_name)\n",
-        "scoring_dataset = Dataset.get_by_name(ws, scoring_dataset_name)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Create datadset monitor for scoring dataset against training dataset\n",
-        "\n",
-        "Check out [datadrift on dataset notebook](../../work-with-data/datadrift-tutorial/datadrift-tutorial.ipynb) for more details"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Create an Azure Machine Learning Compute cluster\n",
-        "\n",
-        "The data drift capability needs a compute target for computing drift and other data metrics. "
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.compute import AmlCompute\n",
-        "\n",
-        "compute_name = 'cpu-cluster'\n",
-        "\n",
-        "if compute_name in ws.compute_targets:\n",
-        "    compute_target = ws.compute_targets[compute_name]\n",
-        "    if compute_target and type(compute_target) is AmlCompute:\n",
-        "        print('found compute target. just use it. ' + compute_name)\n",
-        "else:\n",
-        "    print('creating a new compute target...')\n",
-        "    provisioning_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D3_V2', min_nodes=0, max_nodes=2)\n",
-        "\n",
-        "    # create the cluster\n",
-        "    compute_target = ComputeTarget.create(ws, compute_name, provisioning_config)\n",
-        "\n",
-        "    # can poll for a minimum number of nodes and for a specific timeout.\n",
-        "    # if no min node count is provided it will use the scale settings for the cluster\n",
-        "    compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)\n",
-        "\n",
-        "    # For a more detailed view of current AmlCompute status, use get_status()\n",
-        "    print(compute_target.get_status().serialize())"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Create and update the data drift object"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.datadrift import DataDriftDetector, AlertConfiguration\n",
-        "\n",
-        "alert_config = AlertConfiguration(['user@contoso.com']) # replace with your email to recieve alerts from the scheduled pipeline after enabling\n",
-        "monitor_name = \"monitor_model_demo\"\n",
-        "baseline = dset # training dataset\n",
-        "target = scoring_dataset # scording dataset\n",
-        "\n",
-        "try:\n",
-        "    monitor = DataDriftDetector.create_from_datasets(ws, monitor_name, baseline, target, \n",
-        "                                                          compute_target='cpu-cluster',         # compute target for scheduled pipeline and backfills \n",
-        "                                                          frequency='Day',                     # how often to analyze target data\n",
-        "                                                          feature_list=None,                    # list of features to detect drift on\n",
-        "                                                          drift_threshold=None,                 # threshold from 0 to 1 for email alerting\n",
-        "                                                          latency=0,                            # SLA in hours for target data to arrive in the dataset\n",
-        "                                                          alert_config=alert_config)            # email addresses to send alert\n",
-        "except KeyError:\n",
-        "    monitor = DataDriftDetector.get_by_name(ws, monitor_name)\n",
-        "    \n",
-        "monitor"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# many monitor settings can be updated \n",
-        "monitor = monitor.update(drift_threshold = 0.1, feature_list = X_features)\n",
-        "\n",
-        "monitor"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Analyze today's scoring data\n",
-        "\n",
-        "Perform a data drift run on the data sent to the service earlier in this notebook. If you set your email address in the alert configuration and the drift threshold <=0.1 you should recieve an email alert to drift from this run.\n",
-        "\n",
-        "Wait for the run to complete before getting the results. "
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from datetime import datetime\n",
-        "\n",
-        "now = datetime.utcnow()\n",
-        "target_date = datetime(now.year, now.month, now.day)\n",
-        "analysis_run = monitor.backfill(target_date, target_date)\n",
-        "analysis_run"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Query metrics and show results in Python\n",
-        "\n",
-        "The below cell will plot some key data drift metrics, and can be used to query the results. Run `help(monitor.get_output)` for specifics on the object returned."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "analysis_run.wait_for_completion(wait_post_processing=True)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "### Get and view results and metrics\n",
-        "\n",
-        "For enterprise workspaces, the UI in the Azure Machine Learning studio can be used. Otherwise, the metrics can be queried in Python and plotted. "
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# get results from Python SDK after the analysis run finishes\n",
-        "results, metrics = monitor.get_output(start_time=target_date, end_time=target_date)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# plot the results from Python SDK \n",
-        "monitor.show(start_time=target_date, end_time=target_date)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Enable the monitor's pipeline schedule\n",
-        "\n",
-        "Turn on a scheduled pipeline which will anlayze the serving dataset for drift. "
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "# enable the pipeline schedule and recieve email alerts\n",
-        "monitor.enable_schedule()\n",
-        "\n",
-        "# disable the pipeline schedule \n",
-        "#monitor.disable_schedule()"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Delete the DataDriftDetector\n",
-        "\n",
-        "Invoking the `delete()` method on the object deletes the the drift monitor permanently and cannot be undone. You will no longer be able to find it in the UI and the `list()` or `get()` methods. The object on which delete() was called will have its state set to deleted and name suffixed with deleted. The baseline and target datasets and model data that was collected, if any, are not deleted. The compute is not deleted. The DataDrift schedule pipeline is disabled and archived."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "monitor.delete()"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "## Next steps\n",
-        "\n",
-        "  * See [our documentation](https://aka.ms/datadrift/aks) or [Python SDK reference](https://docs.microsoft.com/python/api/overview/azure/ml/intro)\n",
-        "  * [Send requests or feedback](mailto:driftfeedback@microsoft.com) on data drift directly to the team\n",
-        "  * Please open issues with data drift here on GitHub or on StackOverflow if others are likely to run into the same issue"
-      ]
-    }
-  ],
-  "metadata": {
-    "authors": [
-      {
-        "name": "jamgan"
-      }
-    ],
-    "category": "tutorial",
-    "compute": [
-      "Remote"
-    ],
-    "datasets": [
-      "NOAA"
-    ],
-    "deployment": [
-      "AKS"
-    ],
-    "exclude_from_index": false,
-    "framework": [
-      "Azure ML"
-    ],
-    "friendly_name": "Data drift on aks",
-    "index_order": 1,
-    "kernelspec": {
-      "display_name": "Python 3.6",
-      "language": "python",
-      "name": "python36"
-    },
-    "language_info": {
-      "codemirror_mode": {
-        "name": "ipython",
-        "version": 3
-      },
-      "file_extension": ".py",
-      "mimetype": "text/x-python",
-      "name": "python",
-      "nbconvert_exporter": "python",
-      "pygments_lexer": "ipython3",
-      "version": "3.6.10"
-    },
-    "star_tag": [
-      "featured"
-    ],
-    "tags": [
-      "Dataset",
-      "Timeseries",
-      "Drift"
-    ],
-    "task": "Filtering"
-  },
-  "nbformat": 4,
-  "nbformat_minor": 4
-}

how-to-use-azureml/monitor-models/data-drift/drift-on-aks.yml

@@ -1,8 +0,0 @@
-name: drift-on-aks
-dependencies:
-- pip:
-  - azureml-sdk
-  - azureml-datadrift
-  - azureml-monitoring
-  - azureml-opendatasets
-  - azureml-widgets

how-to-use-azureml/monitor-models/data-drift/myenv.yml

@@ -1,11 +0,0 @@
-name: project_environment
-dependencies:
-  - python=3.6.2
-  - pip:
-    - azureml-core
-    - azureml-defaults
-    - azureml-monitoring
-    - scikit-learn
-    - numpy
-    - packaging
-    - inference-schema[numpy-support]

how-to-use-azureml/monitor-models/data-drift/score.py

@@ -1,44 +0,0 @@
-import os
-
-import numpy as np
-from azureml.monitoring import ModelDataCollector
-from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType
-from inference_schema.schema_decorators import input_schema, output_schema
-# sklearn.externals.joblib is removed in 0.23
-from sklearn import __version__ as sklearnver
-from packaging.version import Version
-if Version(sklearnver) < Version("0.23.0"):
-    from sklearn.externals import joblib
-else:
-    import joblib
-
-
-def init():
-    global model
-    global inputs_dc
-    inputs_dc = ModelDataCollector('elevation-regression-model.pkl', designation='inputs',
-                                   feature_names=['latitude', 'longitude', 'temperature', 'windAngle', 'windSpeed'])
-    # note here "elevation-regression-model.pkl" is the name of the model registered under
-    # this is a different behavior than before when the code is run locally, even though the code is the same.
-    # AZUREML_MODEL_DIR is an environment variable created during deployment.
-    # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)
-    # For multiple models, it points to the folder containing all deployed models (./azureml-models)
-    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'elevation-regression-model.pkl')
-    model = joblib.load(model_path)
-
-
-input_sample = np.array([[30, -85, 21, 150, 6]])
-output_sample = np.array([8.995])
-
-
-@input_schema('data', NumpyParameterType(input_sample))
-@output_schema(NumpyParameterType(output_sample))
-def run(data):
-    try:
-        inputs_dc.collect(data)
-        result = model.predict(data)
-        # you can return any datatype as long as it is JSON-serializable
-        return result.tolist()
-    except Exception as e:
-        error = str(e)
-        return error

how-to-use-azureml/reinforcement-learning/minecraft-on-distributed-compute/minecraft.ipynb

@@ -539,7 +539,7 @@
         "    except StopIteration:\n",
         "        time.sleep(1)\n",
         "\n",
-        "tb = Tensorboard([head_run])\n",
+        "tb = Tensorboard([head_run], port=6007)\n",
         "tb.start()"
       ]
     },

how-to-use-azureml/track-and-monitor-experiments/README.md

@@ -12,8 +12,8 @@
 Try out the sample notebooks:
 1. [Use MLflow with Azure Machine Learning for Local Training Run](./using-mlflow/train-local/train-local.ipynb)
 1. [Use MLflow with Azure Machine Learning for Remote Training Run](./using-mlflow/train-remote/train-remote.ipynb)
-1. [Train and Deploy PyTorch Image Classifier](./using-mlflow/train-and-deploy-pytorch/train-and-deploy-pytorch.ipynb)
-1. [Train and Deploy Keras Image Classifier with MLflow auto logging](./using-mlflow/train-and-deploy-keras-auto-logging/train-and-deploy-keras-auto-logging.ipynb)
+1. [Use MLflow with Azure Machine Learning to submit runs locally with MLflow projects](./using-mlflow/train-projects-local/train-projects-local.ipynb)
+1. [Use MLflow with Azure Machine Learning to submit runs on AzureML compute with MLflow projects](./using-mlflow/train-projects-remote/train-projects-remote.ipynb)
 
  ![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/track-and-monitor-experiments/README.png)
 

how-to-use-azureml/track-and-monitor-experiments/logging-api/logging-api.ipynb

@@ -100,7 +100,7 @@
         "\n",
         "# Check core SDK version number\n",
         "\n",
-        "print(\"This notebook was created using SDK version 1.12.0, you are currently running version\", azureml.core.VERSION)"
+        "print(\"This notebook was created using SDK version 1.14.0, you are currently running version\", azureml.core.VERSION)"
       ]
     },
     {

how-to-use-azureml/track-and-monitor-experiments/tensorboard/tensorboard.ipynb

@@ -161,7 +161,7 @@
         "\n",
         "We'll start by running this locally. While it might not initially seem that useful to use this for a local run - why not just run TB against the files generated locally? - even in this case there is some value to using this feature. Your local run will be registered in the run history, and your Tensorboard logs will be uploaded to the artifact store associated with this run. Later, you'll be able to restore the logs from any run, regardless of where it happened.\n",
         "\n",
-        "Note that for this run, you will need to install Tensorflow on your local machine by yourself. Further, the Tensorboard module (that is, the one included with Tensorflow) must be accessible to this notebook's kernel, as the local machine is what runs Tensorboard."
+        "Note that for this run, you will need to install Tensorflow on your local machine by yourself. Further, the Tensorboard module (that is, the one included with Tensorflow) must be accessible to this notebook's kernel, as the local machine is what runs Tensorboard. In addition, you will also need to have the `azureml-tensorboard` package installed."
       ]
     },
     {
@@ -170,14 +170,13 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "from azureml.core.runconfig import RunConfiguration\n",
+        "from azureml.core import Environment\n",
         "\n",
-        "# Create a run configuration.\n",
-        "run_config = RunConfiguration()\n",
-        "run_config.environment.python.user_managed_dependencies = True\n",
+        "myenv = Environment(\"myenv\")\n",
+        "myenv.python.user_managed_dependencies = True\n",
         "\n",
         "# You can choose a specific Python environment by pointing to a Python path \n",
-        "#run_config.environment.python.interpreter_path = '/home/ninghai/miniconda3/envs/sdk2/bin/python'"
+        "#myenv.python.interpreter_path = '/home/ninghai/miniconda3/envs/sdk2/bin/python'"
       ]
     },
     {
@@ -199,20 +198,20 @@
         "\n",
         "# Writing logs to ./logs results in their being uploaded to Artifact Service,\n",
         "# and thus, made accessible to our Tensorboard instance.\n",
-        "arguments_list = [\"--log_dir\", logs_dir]\n",
+        "arguments = [\"--log_dir\", logs_dir]\n",
         "\n",
         "# Create an experiment\n",
         "exp = Experiment(ws, experiment_name)\n",
         "\n",
         "# If you would like the run to go for longer, add --max_steps 5000 to the arguments list:\n",
-        "# arguments_list += [\"--max_steps\", \"5000\"]\n",
+        "# arguments += [\"--max_steps\", \"5000\"]\n",
         "\n",
-        "script = ScriptRunConfig(exp_dir,\n",
-        "                         script=\"mnist_with_summaries.py\",\n",
-        "                         run_config=run_config,\n",
-        "                         arguments=arguments_list)\n",
+        "src = ScriptRunConfig(exp_dir,\n",
+        "                      script=\"mnist_with_summaries.py\",\n",
+        "                      arguments=arguments,\n",
+        "                      environment=myenv)\n",
         "\n",
-        "run = exp.submit(script)\n",
+        "run = exp.submit(src)\n",
         "# You can also wait for the run to complete\n",
         "# run.wait_for_completion(show_output=True)\n",
         "runs.append(run)"
@@ -578,7 +577,7 @@
       "name": "python",
       "nbconvert_exporter": "python",
       "pygments_lexer": "ipython3",
-      "version": "3.6.6"
+      "version": "3.7.7"
     },
     "tags": [
       "None"

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-and-deploy-keras-auto-logging/train-and-deploy-keras-auto-logging.yml

@@ -1,9 +0,0 @@
-name: train-and-deploy-keras-auto-logging
-dependencies:
-- pip:
-  - azureml-sdk
-  - numpy
-  - azureml-mlflow
-  - matplotlib
-  - tensorflow==2.1
-  - keras

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-and-deploy-pytorch/train-and-deploy-pytorch.yml

@@ -1,9 +0,0 @@
-name: train-and-deploy-pytorch
-dependencies:
-- pytorch==1.4.0 -c pytorch
-- torchvision -c pytorch
-- pip:
-  - azureml-sdk
-  - numpy
-  - azureml-mlflow
-  - matplotlib

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/MLproject

@@ -0,0 +1,10 @@
+name: mlflow-example
+
+conda_env: conda.yaml
+
+entry_points:
+  main:
+    parameters:
+      alpha: float
+      l1_ratio: {type: float, default: 0.1}
+    command: "python train.py {alpha} {l1_ratio}"

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/backend_config.json

@@ -0,0 +1,3 @@
+{
+    "USE_CONDA": "False"
+}

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/conda.yaml

@@ -0,0 +1,13 @@
+# Copyright (c) Microsoft. All rights reserved.
+#  Licensed under the MIT license.
+name: mlflow-example
+channels:
+  - defaults
+dependencies:
+  - numpy>=1.14.3
+  - pandas>=1.0.0
+  - scikit-learn=0.19.1
+  - pip
+  - pip:
+    - mlflow
+    - azureml-mlflow

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/train-projects-local.ipynb

@@ -0,0 +1,304 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/train-projects-local.png)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Copyright (c) Microsoft Corporation. All rights reserved.\n",
+        "\n",
+        "Licensed under the MIT License."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# Train with MLflow Projects on local compute\n",
+        "\n",
+        "Train MLflow Projects on your machine with local compute and AzureML tracking. In this notebook you will:\n",
+        "\n",
+        "1. Set up MLflow tracking URI to track experiments and metrics in AzureML\n",
+        "2. Create experiment\n",
+        "3. Set up an MLflow project to run on AzureML compute\n",
+        "4. Submit an MLflow project run and view it in an AzureML workspace "
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Prerequisites \n",
+        "\n",
+        "If you are using a Notebook VM, you're all set. Otherwise, go through the [Configuration](../../../../configuration.ipnyb) notebook to set up your Azure Machine Learning workspace and ensure other common prerequisites are met.\n",
+        "\n",
+        "Install azureml-mlflow package before running this notebook. Note that mlflow itself gets installed as dependency if you haven't installed it yet.\n",
+        "\n",
+        "```\n",
+        "pip install azureml-mlflow\n",
+        "```\n",
+        "\n",
+        "This example also uses scikit-learn. Install them using the following:\n",
+        "```\n",
+        "pip install scikit-learn matplotlib\n",
+        "```\n",
+        "\n",
+        "Make sure you have the following before starting the notebook: \n",
+        "- Connected to an AzureML Workspace \n",
+        "- Your local conda environment has the necessary packages needed to run this project\n",
+        "\n",
+        "\n",
+        "\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Set-up\n",
+        "\n",
+        "Configure your workspace and check package versions"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": []
+      },
+      "outputs": [],
+      "source": [
+        "import sys, os\n",
+        "import mlflow\n",
+        "import mlflow.azureml\n",
+        "\n",
+        "import azureml.core\n",
+        "from azureml.core import Workspace\n",
+        "\n",
+        "ws = Workspace.from_config()\n",
+        "\n",
+        "print(\"SDK version:\", azureml.core.VERSION)\n",
+        "print(\"MLflow version:\", mlflow.version.VERSION)\n",
+        "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Initialize Tracking Store and Experiment\n",
+        "\n",
+        "### Set the Tracking Store\n",
+        "Set the MLflow tracking URI to point to your Azure ML Workspace. The subsequent logging calls from MLflow APIs will go to Azure ML services and will be tracked under your Workspace."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": []
+      },
+      "outputs": [],
+      "source": [
+        "mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Create Experiment\n",
+        "Create an Mlflow Experiment to organize your runs. It can be set either by passing the name as a parameter in the mlflow.projects.run call or by the following,"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": []
+      },
+      "outputs": [],
+      "source": [
+        "experiment_name = \"train-project-local\"\n",
+        "mlflow.set_experiment(experiment_name)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Create the Backend Configuration Object\n",
+        "\n",
+        "The backend configuration object will store necesary information for the integration such as the compute target and whether to use your local managed environment or a system managed environment. \n",
+        "\n",
+        "The integration will accept \"COMPUTE\" and \"USE_CONDA\" as parameters where \"COMPUTE\" is set to the name of a remote target (not applicable for this training example) and \"USE_CONDA\" which creates a new environment for the project from the environment configuration file. You must set this to \"False\" and include it in the backend configuration object if you want to use your local environment for the project run. Mlflow accepts a dictionary object or a JSON file."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# dictionary\n",
+        "backend_config = {\"USE_CONDA\": False}\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Add the Integration to your Environment Configuration\n",
+        "\n",
+        "Add the azureml-mlflow package as a pip dependency to your environment configuration file (conda.yaml). The project can run without this addition, but key artifacts and metrics will not be logged to your Workspace. An example conda.yaml file is included in this notebook folder. Adding it to to the file will look like this,\n",
+        "\n",
+        "```\n",
+        "name: mlflow-example\n",
+        "channels:\n",
+        "  - defaults\n",
+        "  - anaconda\n",
+        "  - conda-forge\n",
+        "dependencies:\n",
+        "  - python=3.6\n",
+        "  - scikit-learn=0.19.1\n",
+        "  - pip\n",
+        "  - pip:\n",
+        "    - mlflow\n",
+        "    - azureml-mlflow\n",
+        "```"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## User Managed environment\n",
+        "For using your local conda environment, set `use_conda = False` in the backend_config object. Ensure your local environment has all the necessary packages for running the project and you are specifying the **backend parameter** in any run call to be **\"azureml\"**."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": []
+      },
+      "outputs": [],
+      "source": [
+        "local_env_run = mlflow.projects.run(uri=\".\", \n",
+        "                                    parameters={\"alpha\":0.3},\n",
+        "                                    backend = \"azureml\",\n",
+        "                                    use_conda=False,\n",
+        "                                    backend_config = backend_config)\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "local_env_run"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Note: All these calculations were run on your local machine, in the conda environment you defined above. You can find the results in:\n",
+        "- Your AzureML Experiments (a link to the run will be provided in the console)\n",
+        "- ~/.azureml/envs/azureml_xxxx for the conda environment you just created\n",
+        "- ~/AppData/Local/Temp/azureml_runs/train-on-local_xxxx for the machine learning models you trained (this path may differ depending on the platform you use). This folder also contains\n",
+        "    - Logs (under azureml_logs/)\n",
+        "    - Output pickled files (under outputs/)\n",
+        "    - The configuration files (credentials, local and docker image setups)\n",
+        "    - The train.py and mylib.py scripts\n",
+        "    - The current notebook"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## System Mananged Environment\n",
+        "\n",
+        "Now, instead of managing the setup of the environment yourself, you can ask the system to build a new conda environment for you using the environment configuration file in this project. If a backend configuration object is not provided in the call, the integration will default to creating a new conda environment. The environment is built once, and will be reused in subsequent executions as long as the conda dependencies remain unchanged.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "tags": []
+      },
+      "outputs": [],
+      "source": [
+        "backend_config = {\"USE_CONDA\": True}\n",
+        "local_mlproject_run = mlflow.projects.run(uri=\".\", \n",
+        "                                    parameters={\"alpha\":0.3},\n",
+        "                                    backend = \"azureml\",\n",
+        "                                    backend_config = backend_config)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Next Steps \n",
+        "\n",
+        "Try out these notebooks to learn more about MLflow-Azure Machine Learning integration:\n",
+        "\n",
+        " * [Train a model using remote compute on Azure Cloud](../train-on-remote/train-on-remote.ipynb)\n",
+        " * [Deploy the model as a web service](../deploy-model/deploy-model.ipynb)\n",
+        " * [Train a model using Pytorch and MLflow](../../ml-frameworks/using-mlflow/train-and-deploy-pytorch)\n",
+        "\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "authors": [
+      {
+        "name": "shipatel"
+      }
+    ],
+    "category": "tutorial",
+    "celltoolbar": "Edit Metadata",
+    "compute": [
+      "Local"
+    ],
+    "exclude_from_index": false,
+    "framework": [
+      "ScikitLearn"
+    ],
+    "friendly_name": "Use MLflow projects with Azure Machine Learning to train a model with local compute",
+    "kernelspec": {
+      "display_name": "Python 3.6",
+      "language": "python",
+      "name": "python36"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.8.5-final"
+    },
+    "tags": [
+      "mlflow",
+      "scikit"
+    ],
+    "task": "Use MLflow projects with Azure Machine Learning to train a model using local compute"
+  },
+  "nbformat": 4,
+  "nbformat_minor": 2
+}

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/train-projects-local.yml

@@ -0,0 +1,7 @@
+name: train-projects-local
+dependencies:
+- scikit-learn
+- pip:
+  - azureml-sdk
+  - scikit-learn
+  - azureml-mlflow

how-to-use-azureml/track-and-monitor-experiments/using-mlflow/train-projects-local/train.py

@@ -0,0 +1,64 @@
+# The data set used in this example is from http://archive.ics.uci.edu/ml/datasets/Wine+Quality
+# P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis.
+# Modeling wine preferences by data mining from physicochemical properties.
+# In Decision Support Systems, Elsevier, 47(4):547-553, 2009.
+
+import os
+import warnings
+import sys
+import pandas as pd
+import numpy as np
+from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
+from sklearn.model_selection import train_test_split
+from sklearn.linear_model import ElasticNet
+import mlflow
+import mlflow.sklearn
+
+
+def eval_metrics(actual, pred):
+    rmse = np.sqrt(mean_squared_error(actual, pred))
+    mae = mean_absolute_error(actual, pred)
+    r2 = r2_score(actual, pred)
+    return rmse, mae, r2
+
+
+if __name__ == "__main__":
+    warnings.filterwarnings("ignore")
+    np.random.seed(40)
+
+    # Read the wine-quality csv file (make sure you're running this from the root of MLflow!)
+    wine_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "wine-quality.csv")
+    data = pd.read_csv(wine_path)
+
+    # Split the data into training and test sets. (0.75, 0.25) split.
+    train, test = train_test_split(data)
+
+    # The predicted column is "quality" which is a scalar from [3, 9]
+    train_x = train.drop(["quality"], axis=1)
+    test_x = test.drop(["quality"], axis=1)
+    train_y = train[["quality"]]
+    test_y = test[["quality"]]
+
+    alpha = float(sys.argv[1]) if len(sys.argv) > 1 else 0.5
+    l1_ratio = float(sys.argv[2]) if len(sys.argv) > 2 else 0.5
+
+    with mlflow.start_run():
+        lr = ElasticNet(alpha=alpha, l1_ratio=l1_ratio, random_state=42)
+        lr.fit(train_x, train_y)
+
+        predicted_qualities = lr.predict(test_x)
+
+        (rmse, mae, r2) = eval_metrics(test_y, predicted_qualities)
+
+        print("Elasticnet model (alpha=%f, l1_ratio=%f):" % (alpha, l1_ratio))
+        print("  RMSE: %s" % rmse)
+        print("  MAE: %s" % mae)
+        print("  R2: %s" % r2)
+
+        mlflow.log_param("alpha", alpha)
+        mlflow.log_param("l1_ratio", l1_ratio)
+        mlflow.log_metric("rmse", rmse)
+        mlflow.log_metric("r2", r2)
+        mlflow.log_metric("mae", mae)
+
+        mlflow.sklearn.log_model(lr, "model")