update samples from Release-58 as a part of SDK release

configuration.ipynb

@@ -103,7 +103,7 @@
       "source": [
         "import azureml.core\n",
         "\n",
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

contrib/fairness/fairlearn-azureml-mitigation.ipynb

@@ -50,7 +50,22 @@
         "* `joblib`\n",
         "* `shap`\n",
         "\n",
-        "\n",
+        "Fairlearn relies on features introduced in v0.22.1 of `scikit-learn`. If you have an older version already installed, please uncomment and run the following cell:"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# !pip install --upgrade scikit-learn>=0.22.1"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
         "<a id=\"LoadingData\"></a>\n",
         "## Loading the Data\n",
         "We use the well-known `adult` census dataset, which we load using `shap` (for convenience). We start with a fairly unremarkable set of imports:"

contrib/fairness/fairlearn-azureml-mitigation.yml

@@ -1,8 +0,0 @@
-name: fairlearn-azureml-mitigation
-dependencies:
-- pip:
-  - azureml-sdk
-  - azureml-contrib-fairness
-  - fairlearn==0.4.6
-  - joblib
-  - shap

contrib/fairness/upload-fairness-dashboard.ipynb

@@ -52,9 +52,22 @@
         "* `joblib`\n",
         "* `shap`\n",
         "\n",
-        "\n",
+        "Fairlearn relies on features introduced in v0.22.1 of `scikit-learn`. If you have an older version already installed, please uncomment and run the following cell:"
-        "\n",
+      ]
-        "\n",
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# !pip install --upgrade scikit-learn>=0.22.1"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
         "<a id=\"LoadingData\"></a>\n",
         "## Loading the Data\n",
         "We use the well-known `adult` census dataset, which we load using `shap` (for convenience). We start with a fairly unremarkable set of imports:"

contrib/fairness/upload-fairness-dashboard.yml

@@ -1,8 +0,0 @@
-name: upload-fairness-dashboard
-dependencies:
-- pip:
-  - azureml-sdk
-  - azureml-contrib-fairness
-  - fairlearn==0.4.6
-  - joblib
-  - shap

how-to-use-azureml/automated-machine-learning/README.md

@@ -230,15 +230,6 @@ You may check the version of tensorflow and uninstall as follows
 2) enter `pip freeze` and look for `tensorflow` , if found, the version listed should be < 1.13
 3) If the listed version is a not a supported version,  `pip uninstall tensorflow` in the command shell and enter y for confirmation.
 
-## KeyError: 'brand' when running AutoML on local compute or Azure Databricks cluster**
-If a new environment was created after 10 June 2020 using SDK 1.7.0 or lower, training may fail with the above error due to an update in the py-cpuinfo package. (Environments created on or before 10 June 2020 are unaffected, as well as experiments run on remote compute as cached training images are used.) To work around this issue, either of the two following steps can be taken:
-
-1) Update the SDK version to 1.8.0 or higher (this will also downgrade py-cpuinfo to 5.0.0):
-`pip install --upgrade azureml-sdk[automl]`
-
-2) Downgrade the installed version of py-cpuinfo to 5.0.0:
-`pip install py-cpuinfo==5.0.0`
-
 ## Remote run: DsvmCompute.create fails
 There are several reasons why the DsvmCompute.create can fail.  The reason is usually in the error message but you have to look at the end of the error message for the detailed reason.  Some common reasons are:
 1) `Compute name is invalid, it should start with a letter, be between 2 and 16 character, and only include letters (a-zA-Z), numbers (0-9) and \'-\'.`  Note that underscore is not allowed in the name.

how-to-use-azureml/automated-machine-learning/automl_env.yml

@@ -6,7 +6,7 @@ dependencies:
 - python>=3.5.2,<3.6.8
 - nb_conda
 - matplotlib==2.1.0
-- numpy>=1.16.0,<=1.16.2
+- numpy~=1.16.0
 - cython
 - urllib3<1.24
 - scipy==1.4.1

how-to-use-azureml/automated-machine-learning/automl_env_mac.yml

@@ -7,7 +7,7 @@ dependencies:
 - python>=3.5.2,<3.6.8
 - nb_conda
 - matplotlib==2.1.0
-- numpy>=1.16.0,<=1.16.2
+- numpy~=1.16.0
 - cython
 - urllib3<1.24
 - scipy==1.4.1

how-to-use-azureml/automated-machine-learning/classification-bank-marketing-all-features/auto-ml-classification-bank-marketing-all-features.ipynb

@@ -57,7 +57,7 @@
         "9. Test the ACI service.\n",
         "\n",
         "In addition this notebook showcases the following features\n",
-        "- **Blacklisting** certain pipelines\n",
+        "- **Blocking** certain pipelines\n",
         "- Specifying **target metrics** to indicate stopping criteria\n",
         "- Handling **missing data** in the input"
       ]
@@ -105,7 +105,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -314,8 +314,8 @@
         "|**task**|classification or regression or forecasting|\n",
         "|**primary_metric**|This is the metric that you want to optimize. Classification supports the following primary metrics: <br><i>accuracy</i><br><i>AUC_weighted</i><br><i>average_precision_score_weighted</i><br><i>norm_macro_recall</i><br><i>precision_score_weighted</i>|\n",
         "|**iteration_timeout_minutes**|Time limit in minutes for each iteration.|\n",
-        "|**blacklist_models** | *List* of *strings* indicating machine learning algorithms for AutoML to avoid in this run. <br><br> Allowed values for **Classification**<br><i>LogisticRegression</i><br><i>SGD</i><br><i>MultinomialNaiveBayes</i><br><i>BernoulliNaiveBayes</i><br><i>SVM</i><br><i>LinearSVM</i><br><i>KNN</i><br><i>DecisionTree</i><br><i>RandomForest</i><br><i>ExtremeRandomTrees</i><br><i>LightGBM</i><br><i>GradientBoosting</i><br><i>TensorFlowDNN</i><br><i>TensorFlowLinearClassifier</i><br><br>Allowed values for **Regression**<br><i>ElasticNet</i><br><i>GradientBoosting</i><br><i>DecisionTree</i><br><i>KNN</i><br><i>LassoLars</i><br><i>SGD</i><br><i>RandomForest</i><br><i>ExtremeRandomTrees</i><br><i>LightGBM</i><br><i>TensorFlowLinearRegressor</i><br><i>TensorFlowDNN</i><br><br>Allowed values for **Forecasting**<br><i>ElasticNet</i><br><i>GradientBoosting</i><br><i>DecisionTree</i><br><i>KNN</i><br><i>LassoLars</i><br><i>SGD</i><br><i>RandomForest</i><br><i>ExtremeRandomTrees</i><br><i>LightGBM</i><br><i>TensorFlowLinearRegressor</i><br><i>TensorFlowDNN</i><br><i>Arima</i><br><i>Prophet</i>|\n",
+        "|**blocked_models** | *List* of *strings* indicating machine learning algorithms for AutoML to avoid in this run. <br><br> Allowed values for **Classification**<br><i>LogisticRegression</i><br><i>SGD</i><br><i>MultinomialNaiveBayes</i><br><i>BernoulliNaiveBayes</i><br><i>SVM</i><br><i>LinearSVM</i><br><i>KNN</i><br><i>DecisionTree</i><br><i>RandomForest</i><br><i>ExtremeRandomTrees</i><br><i>LightGBM</i><br><i>GradientBoosting</i><br><i>TensorFlowDNN</i><br><i>TensorFlowLinearClassifier</i><br><br>Allowed values for **Regression**<br><i>ElasticNet</i><br><i>GradientBoosting</i><br><i>DecisionTree</i><br><i>KNN</i><br><i>LassoLars</i><br><i>SGD</i><br><i>RandomForest</i><br><i>ExtremeRandomTrees</i><br><i>LightGBM</i><br><i>TensorFlowLinearRegressor</i><br><i>TensorFlowDNN</i><br><br>Allowed values for **Forecasting**<br><i>ElasticNet</i><br><i>GradientBoosting</i><br><i>DecisionTree</i><br><i>KNN</i><br><i>LassoLars</i><br><i>SGD</i><br><i>RandomForest</i><br><i>ExtremeRandomTrees</i><br><i>LightGBM</i><br><i>TensorFlowLinearRegressor</i><br><i>TensorFlowDNN</i><br><i>Arima</i><br><i>Prophet</i>|\n",
-        "| **whitelist_models** |  *List* of *strings* indicating machine learning algorithms for AutoML to use in this run. Same values listed above for **blacklist_models** allowed for **whitelist_models**.|\n",
+        "|**allowed_models** |  *List* of *strings* indicating machine learning algorithms for AutoML to use in this run. Same values listed above for **blocked_models** allowed for **allowed_models**.|\n",
         "|**experiment_exit_score**| Value indicating the target for *primary_metric*. <br>Once the target is surpassed the run terminates.|\n",
         "|**experiment_timeout_hours**| Maximum amount of time in hours that all iterations combined can take before the experiment terminates.|\n",
         "|**enable_early_stopping**| Flag to enble early termination if the score is not improving in the short term.|\n",
@@ -349,7 +349,7 @@
         "                             debug_log = 'automl_errors.log',\n",
         "                             compute_target=compute_target,\n",
         "                             experiment_exit_score = 0.9984,\n",
-        "                             blacklist_models = ['KNN','LinearSVM'],\n",
+        "                             blocked_models = ['KNN','LinearSVM'],\n",
         "                             enable_onnx_compatible_models=True,\n",
         "                             training_data = train_data,\n",
         "                             label_column_name = label,\n",

how-to-use-azureml/automated-machine-learning/classification-credit-card-fraud/auto-ml-classification-credit-card-fraud.ipynb

@@ -93,7 +93,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/classification-text-dnn/auto-ml-classification-text-dnn.ipynb

@@ -97,7 +97,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/continuous-retraining/auto-ml-continuous-retraining.ipynb

@@ -88,7 +88,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-beer-remote/auto-ml-forecasting-beer-remote.ipynb

@@ -114,7 +114,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -352,8 +352,6 @@
         "|**label_column_name**|The name of the label column.|\n",
         "|**enable_dnn**|Enable Forecasting DNNs|\n",
         "\n",
-        "This notebook uses the blacklist_models parameter to exclude some models that take a longer time to train on this dataset. You can choose to remove models from the blacklist_models list but you may need to increase the iteration_timeout_minutes parameter value to get results.\n",
-        "\n",
         "This step requires an Enterprise workspace to gain access to this feature. To learn more about creating an Enterprise workspace or upgrading to an Enterprise workspace from the Azure portal, please visit our [Workspace page.](https://docs.microsoft.com/azure/machine-learning/service/concept-workspace#upgrade)."
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-bike-share/auto-ml-forecasting-bike-share.ipynb

@@ -87,7 +87,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -250,7 +250,7 @@
         "|-|-|\n",
         "|**task**|forecasting|\n",
         "|**primary_metric**|This is the metric that you want to optimize.<br> Forecasting supports the following primary metrics <br><i>spearman_correlation</i><br><i>normalized_root_mean_squared_error</i><br><i>r2_score</i><br><i>normalized_mean_absolute_error</i>\n",
-        "|**blacklist_models**|Models in blacklist won't be used by AutoML. All supported models can be found at [here](https://docs.microsoft.com/en-us/python/api/azureml-train-automl-client/azureml.train.automl.constants.supportedmodels.forecasting?view=azure-ml-py).|\n",
+        "|**blocked_models**|Models in blocked_models won't be used by AutoML. All supported models can be found at [here](https://docs.microsoft.com/en-us/python/api/azureml-train-automl-client/azureml.train.automl.constants.supportedmodels.forecasting?view=azure-ml-py).|\n",
         "|**experiment_timeout_hours**|Experimentation timeout in hours.|\n",
         "|**training_data**|Input dataset, containing both features and label column.|\n",
         "|**label_column_name**|The name of the label column.|\n",
@@ -263,7 +263,7 @@
         "|**target_lags**|The target_lags specifies how far back we will construct the lags of the target variable.|\n",
         "|**drop_column_names**|Name(s) of columns to drop prior to modeling|\n",
         "\n",
-        "This notebook uses the blacklist_models parameter to exclude some models that take a longer time to train on this dataset. You can choose to remove models from the blacklist_models list but you may need to increase the experiment_timeout_hours parameter value to get results."
+        "This notebook uses the blocked_models parameter to exclude some models that take a longer time to train on this dataset. You can choose to remove models from the blocked_models list but you may need to increase the experiment_timeout_hours parameter value to get results."
       ]
     },
     {
@@ -307,7 +307,7 @@
         "\n",
         "automl_config = AutoMLConfig(task='forecasting',                             \n",
         "                             primary_metric='normalized_root_mean_squared_error',\n",
-        "                             blacklist_models = ['ExtremeRandomTrees'],                             \n",
+        "                             blocked_models = ['ExtremeRandomTrees'],                             \n",
         "                             experiment_timeout_hours=0.3,\n",
         "                             training_data=train,\n",
         "                             label_column_name=target_column_name,\n",

how-to-use-azureml/automated-machine-learning/forecasting-energy-demand/auto-ml-forecasting-energy-demand.ipynb

@@ -97,7 +97,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },
@@ -303,7 +303,7 @@
         "|-|-|\n",
         "|**task**|forecasting|\n",
         "|**primary_metric**|This is the metric that you want to optimize.<br> Forecasting supports the following primary metrics <br><i>spearman_correlation</i><br><i>normalized_root_mean_squared_error</i><br><i>r2_score</i><br><i>normalized_mean_absolute_error</i>|\n",
-        "|**blacklist_models**|Models in blacklist won't be used by AutoML. All supported models can be found at [here](https://docs.microsoft.com/en-us/python/api/azureml-train-automl-client/azureml.train.automl.constants.supportedmodels.forecasting?view=azure-ml-py).|\n",
+        "|**blocked_models**|Models in blocked_models won't be used by AutoML. All supported models can be found at [here](https://docs.microsoft.com/en-us/python/api/azureml-train-automl-client/azureml.train.automl.constants.supportedmodels.forecasting?view=azure-ml-py).|\n",
         "|**experiment_timeout_hours**|Maximum amount of time in hours that the experiment take before it terminates.|\n",
         "|**training_data**|The training data to be used within the experiment.|\n",
         "|**label_column_name**|The name of the label column.|\n",
@@ -318,7 +318,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "This notebook uses the blacklist_models parameter to exclude some models that take a longer time to train on this dataset. You can choose to remove models from the blacklist_models list but you may need to increase the experiment_timeout_hours parameter value to get results."
+        "This notebook uses the blocked_models parameter to exclude some models that take a longer time to train on this dataset. You can choose to remove models from the blocked_models list but you may need to increase the experiment_timeout_hours parameter value to get results."
       ]
     },
     {
@@ -334,7 +334,7 @@
         "\n",
         "automl_config = AutoMLConfig(task='forecasting',                             \n",
         "                             primary_metric='normalized_root_mean_squared_error',\n",
-        "                             blacklist_models = ['ExtremeRandomTrees', 'AutoArima', 'Prophet'],                             \n",
+        "                             blocked_models = ['ExtremeRandomTrees', 'AutoArima', 'Prophet'],                             \n",
         "                             experiment_timeout_hours=0.3,\n",
         "                             training_data=train,\n",
         "                             label_column_name=target_column_name,\n",
@@ -560,7 +560,7 @@
         "### Using lags and rolling window features\n",
         "Now we will configure the target lags, that is the previous values of the target variables, meaning the prediction is no longer horizon-less. We therefore must still specify the `max_horizon` that the model will learn to forecast. The `target_lags` keyword specifies how far back we will construct the lags of the target variable, and the `target_rolling_window_size` specifies the size of the rolling window over which we will generate the `max`, `min` and `sum` features.\n",
         "\n",
-        "This notebook uses the blacklist_models parameter to exclude some models that take a longer time to train on this dataset.  You can choose to remove models from the blacklist_models list but you may need to increase the iteration_timeout_minutes parameter value to get results."
+        "This notebook uses the blocked_models parameter to exclude some models that take a longer time to train on this dataset.  You can choose to remove models from the blocked_models list but you may need to increase the iteration_timeout_minutes parameter value to get results."
       ]
     },
     {
@@ -578,7 +578,7 @@
         "\n",
         "automl_config = AutoMLConfig(task='forecasting',                             \n",
         "                             primary_metric='normalized_root_mean_squared_error',\n",
-        "                             blacklist_models = ['ElasticNet','ExtremeRandomTrees','GradientBoosting','XGBoostRegressor','ExtremeRandomTrees', 'AutoArima', 'Prophet'], #These models are blacklisted for tutorial purposes, remove this for real use cases.                            \n",
+        "                             blocked_models = ['ElasticNet','ExtremeRandomTrees','GradientBoosting','XGBoostRegressor','ExtremeRandomTrees', 'AutoArima', 'Prophet'], #These models are blocked for tutorial purposes, remove this for real use cases.                            \n",
         "                             experiment_timeout_hours=0.3,\n",
         "                             training_data=train,\n",
         "                             label_column_name=target_column_name,\n",

how-to-use-azureml/automated-machine-learning/forecasting-forecast-function/auto-ml-forecasting-function.ipynb

@@ -94,7 +94,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/forecasting-orange-juice-sales/auto-ml-forecasting-orange-juice-sales.ipynb

@@ -82,7 +82,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/local-run-classification-credit-card-fraud/auto-ml-classification-credit-card-fraud-local.ipynb

@@ -96,7 +96,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/regression-explanation-featurization/auto-ml-regression-explanation-featurization.ipynb

@@ -98,7 +98,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/automated-machine-learning/regression/auto-ml-regression.ipynb

@@ -92,7 +92,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

how-to-use-azureml/deployment/production-deploy-to-aks/production-deploy-to-aks-ssl.ipynb

@@ -149,7 +149,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "%%writefile score.py\n",
+        "%%writefile score_ssl.py\n",
         "import os\n",
         "import pickle\n",
         "import json\n",
@@ -201,7 +201,7 @@
       "source": [
         "from azureml.core.model import InferenceConfig\n",
         "\n",
-        "inf_config = InferenceConfig(entry_script='score.py', environment=myenv)"
+        "inf_config = InferenceConfig(entry_script='score_ssl.py', environment=myenv)"
       ]
     },
     {

how-to-use-azureml/deployment/production-deploy-to-aks/production-deploy-to-aks.ipynb

@@ -109,7 +109,7 @@
         "from azureml.core import Environment\n",
         "from azureml.core.conda_dependencies import CondaDependencies \n",
         "\n",
-        "conda_deps = CondaDependencies.create(conda_packages=['numpy','scikit-learn==0.19.1','scipy'], pip_packages=['azureml-defaults'])\n",
+        "conda_deps = CondaDependencies.create(conda_packages=['numpy','scikit-learn==0.19.1','scipy'], pip_packages=['azureml-defaults', 'inference-schema'])\n",
         "myenv = Environment(name='myenv')\n",
         "myenv.python.conda_dependencies = conda_deps"
       ]

how-to-use-azureml/explain-model/azure-integration/remote-explanation/explain-model-on-amlcompute.ipynb

@@ -204,108 +204,9 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "### Option 1: Provision as a run based compute target\n",
+        "### Option 1: Provision a compute target (Basic)\n",
         "\n",
-        "You can provision AmlCompute as a compute target at run-time. In this case, the compute is auto-created for your run, scales up to max_nodes that you specify, and then **deleted automatically** after the run completes."
+        "You can provision an AmlCompute resource by simply defining two parameters thanks to smart defaults. By default it autoscales from 0 nodes and provisions dedicated VMs to run your job in a container. This is useful when you want to continously re-use the same target, debug it between jobs or simply share the resource with other users of your workspace.\n",
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.core.runconfig import RunConfiguration\n",
-        "from azureml.core.conda_dependencies import CondaDependencies\n",
-        "from azureml.core.runconfig import DEFAULT_CPU_IMAGE\n",
-        "\n",
-        "# create a new runconfig object\n",
-        "run_config = RunConfiguration()\n",
-        "\n",
-        "# signal that you want to use AmlCompute to execute script.\n",
-        "run_config.target = \"amlcompute\"\n",
-        "\n",
-        "# AmlCompute will be created in the same region as workspace\n",
-        "# Set vm size for AmlCompute\n",
-        "run_config.amlcompute.vm_size = 'STANDARD_D2_V2'\n",
-        "\n",
-        "# enable Docker \n",
-        "run_config.environment.docker.enabled = True\n",
-        "\n",
-        "# set Docker base image to the default CPU-based image\n",
-        "run_config.environment.docker.base_image = DEFAULT_CPU_IMAGE\n",
-        "\n",
-        "# use conda_dependencies.yml to create a conda environment in the Docker image for execution\n",
-        "run_config.environment.python.user_managed_dependencies = False\n",
-        "\n",
-        "azureml_pip_packages = [\n",
-        "    'azureml-defaults', 'azureml-contrib-interpret', 'azureml-core', 'azureml-telemetry',\n",
-        "    'azureml-interpret', 'sklearn-pandas', 'azureml-dataprep'\n",
-        "]\n",
-        "\n",
-        "# Note: this is to pin the scikit-learn and pandas versions to be same as notebook.\n",
-        "# In production scenario user would choose their dependencies\n",
-        "import pkg_resources\n",
-        "available_packages = pkg_resources.working_set\n",
-        "sklearn_ver = None\n",
-        "pandas_ver = None\n",
-        "for dist in available_packages:\n",
-        "    if dist.key == 'scikit-learn':\n",
-        "        sklearn_ver = dist.version\n",
-        "    elif dist.key == 'pandas':\n",
-        "        pandas_ver = dist.version\n",
-        "sklearn_dep = 'scikit-learn'\n",
-        "pandas_dep = 'pandas'\n",
-        "if sklearn_ver:\n",
-        "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
-        "if pandas_ver:\n",
-        "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
-        "# specify CondaDependencies obj\n",
-        "# The CondaDependencies specifies the conda and pip packages that are installed in the environment\n",
-        "# the submitted job is run in.  Note the remote environment(s) needs to be similar to the local\n",
-        "# environment, otherwise if a model is trained or deployed in a different environment this can\n",
-        "# cause errors.  Please take extra care when specifying your dependencies in a production environment.\n",
-        "run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=[sklearn_dep, pandas_dep],\n",
-        "                                                                            pip_packages=azureml_pip_packages)\n",
-        "\n",
-        "# Now submit a run on AmlCompute\n",
-        "from azureml.core.script_run_config import ScriptRunConfig\n",
-        "\n",
-        "script_run_config = ScriptRunConfig(source_directory=project_folder,\n",
-        "                                    script='train_explain.py',\n",
-        "                                    run_config=run_config)\n",
-        "\n",
-        "run = experiment.submit(script_run_config)\n",
-        "\n",
-        "# Show run details\n",
-        "run"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "Note: if you need to cancel a run, you can follow [these instructions](https://aka.ms/aml-docs-cancel-run)."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "%%time\n",
-        "# Shows output of the run on stdout.\n",
-        "run.wait_for_completion(show_output=True)"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {},
-      "source": [
-        "### Option 2: Provision as a persistent compute target (Basic)\n",
-        "\n",
-        "You can provision a persistent AmlCompute resource by simply defining two parameters thanks to smart defaults. By default it autoscales from 0 nodes and provisions dedicated VMs to run your job in a container. This is useful when you want to continously re-use the same target, debug it between jobs or simply share the resource with other users of your workspace.\n",
         "\n",
         "* `vm_size`: VM family of the nodes provisioned by AmlCompute. Simply choose from the supported_vmsizes() above\n",
         "* `max_nodes`: Maximum nodes to autoscale to while running a job on AmlCompute"
@@ -351,13 +252,13 @@
         "from azureml.core.runconfig import RunConfiguration\n",
         "from azureml.core.conda_dependencies import CondaDependencies\n",
         "\n",
-        "# create a new RunConfig object\n",
+        "# Create a new RunConfig object\n",
         "run_config = RunConfiguration(framework=\"python\")\n",
         "\n",
         "# Set compute target to AmlCompute target created in previous step\n",
         "run_config.target = cpu_cluster.name\n",
         "\n",
-        "# enable Docker \n",
+        "# Enable Docker \n",
         "run_config.environment.docker.enabled = True\n",
         "\n",
         "azureml_pip_packages = [\n",
@@ -382,7 +283,7 @@
         "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
         "if pandas_ver:\n",
         "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
-        "# specify CondaDependencies obj\n",
+        "# Specify CondaDependencies obj\n",
         "# The CondaDependencies specifies the conda and pip packages that are installed in the environment\n",
         "# the submitted job is run in.  Note the remote environment(s) needs to be similar to the local\n",
         "# environment, otherwise if a model is trained or deployed in a different environment this can\n",
@@ -400,6 +301,13 @@
         "run"
       ]
     },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Note: if you need to cancel a run, you can follow [these instructions](https://aka.ms/aml-docs-cancel-run)."
+      ]
+    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -424,7 +332,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "### Option 3: Provision as a persistent compute target (Advanced)\n",
+        "### Option 2: Provision a compute target (Advanced)\n",
         "\n",
         "You can also specify additional properties or change defaults while provisioning AmlCompute using a more advanced configuration. This is useful when you want a dedicated cluster of 4 nodes (for example you can set the min_nodes and max_nodes to 4), or want the compute to be within an existing VNet in your subscription.\n",
         "\n",
@@ -483,13 +391,13 @@
         "from azureml.core.runconfig import RunConfiguration\n",
         "from azureml.core.conda_dependencies import CondaDependencies\n",
         "\n",
-        "# create a new RunConfig object\n",
+        "# Create a new RunConfig object\n",
         "run_config = RunConfiguration(framework=\"python\")\n",
         "\n",
         "# Set compute target to AmlCompute target created in previous step\n",
         "run_config.target = cpu_cluster.name\n",
         "\n",
-        "# enable Docker \n",
+        "# Enable Docker \n",
         "run_config.environment.docker.enabled = True\n",
         "\n",
         "azureml_pip_packages = [\n",
@@ -516,7 +424,7 @@
         "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
         "if pandas_ver:\n",
         "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
-        "# specify CondaDependencies obj\n",
+        "# Specify CondaDependencies obj\n",
         "# The CondaDependencies specifies the conda and pip packages that are installed in the environment\n",
         "# the submitted job is run in.  Note the remote environment(s) needs to be similar to the local\n",
         "# environment, otherwise if a model is trained or deployed in a different environment this can\n",
@@ -554,19 +462,6 @@
         "run.get_metrics()"
       ]
     },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {},
-      "outputs": [],
-      "source": [
-        "from azureml.contrib.interpret.explanation.explanation_client import ExplanationClient\n",
-        "\n",
-        "client = ExplanationClient.from_run(run)\n",
-        "# Get the top k (e.g., 4) most important features with their importance values\n",
-        "explanation = client.download_model_explanation(top_k=4)"
-      ]
-    },
     {
       "cell_type": "markdown",
       "metadata": {},
@@ -682,7 +577,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# retrieve model for visualization and deployment\n",
+        "# Retrieve model for visualization and deployment\n",
         "from azureml.core.model import Model\n",
         "import joblib\n",
         "original_model = Model(ws, 'model_explain_model_on_amlcomp')\n",
@@ -703,7 +598,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# retrieve x_test for visualization\n",
+        "# Retrieve x_test for visualization\n",
         "import joblib\n",
         "x_test_path = './x_test_boston_housing.pkl'\n",
         "run.download_file('x_test_boston_housing.pkl', output_file_path=x_test_path)"

how-to-use-azureml/explain-model/azure-integration/run-history/save-retrieve-explanations-run-history.ipynb

@@ -122,7 +122,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# get the IBM employee attrition dataset\n",
+        "# Get the IBM employee attrition dataset\n",
         "outdirname = 'dataset.6.21.19'\n",
         "try:\n",
         "    from urllib import urlretrieve\n",
@@ -163,7 +163,7 @@
         "from sklearn.model_selection import train_test_split\n",
         "x_train, x_test, y_train, y_test = train_test_split(attritionXData, \n",
         "                                                    target, \n",
-        "                                                    test_size = 0.2,\n",
+        "                                                    test_size=0.2,\n",
         "                                                    random_state=0,\n",
         "                                                    stratify=target)"
       ]
@@ -223,7 +223,7 @@
         "# Append classifier to preprocessing pipeline.\n",
         "# Now we have a full prediction pipeline.\n",
         "clf = Pipeline(steps=[('preprocessor', transformations),\n",
-        "                      ('classifier', SVC(kernel='linear', C = 1.0, probability=True))])"
+        "                      ('classifier', SVC(C=1.0, probability=True))])"
       ]
     },
     {
@@ -249,7 +249,7 @@
         "# Append classifier to preprocessing pipeline.\n",
         "# Now we have a full prediction pipeline.\n",
         "clf = Pipeline(steps=[('preprocessor', transformations),\n",
-        "                      ('classifier', SVC(kernel='linear', C = 1.0, probability=True))]) \n",
+        "                      ('classifier', SVC(C=1.0, probability=True))]) \n",
         "\n",
         "\n",
         "\n",
@@ -393,7 +393,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# feature shap values for all features and all data points in the training data\n",
+        "# Feature shap values for all features and all data points in the training data\n",
         "print('local importance values: {}'.format(global_explanation.local_importance_values))"
       ]
     },
@@ -450,8 +450,7 @@
       "outputs": [],
       "source": [
         "import azureml.core\n",
-        "from azureml.core import Workspace, Experiment, Run\n",
+        "from azureml.core import Workspace, Experiment\n",
-        "from interpret.ext.blackbox import TabularExplainer\n",
         "from azureml.contrib.interpret.explanation.explanation_client import ExplanationClient\n",
         "# Check core SDK version number\n",
         "print(\"SDK version:\", azureml.core.VERSION)"
@@ -576,6 +575,23 @@
         "ExplanationDashboard(downloaded_global_explanation, model, datasetX=x_test)"
       ]
     },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## End\n",
+        "Complete the run"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "run.complete()"
+      ]
+    },
     {
       "cell_type": "markdown",
       "metadata": {},

how-to-use-azureml/explain-model/azure-integration/scoring-time/train-explain-model-locally-and-deploy.ipynb

@@ -141,7 +141,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# get IBM attrition data\n",
+        "# Get IBM attrition data\n",
         "import os\n",
         "import pandas as pd\n",
         "\n",
@@ -218,17 +218,17 @@
         "from sklearn.model_selection import train_test_split\n",
         "x_train, x_test, y_train, y_test = train_test_split(attritionXData,\n",
         "                                                    target,\n",
-        "                                                    test_size = 0.2,\n",
+        "                                                    test_size=0.2,\n",
         "                                                    random_state=0,\n",
         "                                                    stratify=target)\n",
         "\n",
-        "# preprocess the data and fit the classification model\n",
+        "# Preprocess the data and fit the classification model\n",
         "clf.fit(x_train, y_train)\n",
         "model = clf.steps[-1][1]\n",
         "\n",
         "model_file_name = 'log_reg.pkl'\n",
         "\n",
-        "# save model in the outputs folder so it automatically get uploaded\n",
+        "# Save model in the outputs folder so it automatically get uploaded\n",
         "with open(model_file_name, 'wb') as file:\n",
         "    joblib.dump(value=clf, filename=os.path.join('./outputs/',\n",
         "                                                 model_file_name))"
@@ -345,7 +345,7 @@
         "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
         "if pandas_ver:\n",
         "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
-        "# specify CondaDependencies obj\n",
+        "# Specify CondaDependencies obj\n",
         "# The CondaDependencies specifies the conda and pip packages that are installed in the environment\n",
         "# the submitted job is run in.  Note the remote environment(s) needs to be similar to the local\n",
         "# environment, otherwise if a model is trained or deployed in a different environment this can\n",
@@ -368,7 +368,7 @@
       "outputs": [],
       "source": [
         "from azureml.core.model import Model\n",
-        "# retrieve scoring explainer for deployment\n",
+        "# Retrieve scoring explainer for deployment\n",
         "scoring_explainer_model = Model(ws, 'IBM_attrition_explainer')"
       ]
     },
@@ -416,11 +416,11 @@
         "\n",
         "headers = {'Content-Type':'application/json'}\n",
         "\n",
-        "# send request to service\n",
+        "# Send request to service\n",
         "print(\"POST to url\", service.scoring_uri)\n",
         "resp = requests.post(service.scoring_uri, sample_data, headers=headers)\n",
         "\n",
-        "# can covert back to Python objects from json string if desired\n",
+        "# Can covert back to Python objects from json string if desired\n",
         "print(\"prediction:\", resp.text)\n",
         "result = json.loads(resp.text)"
       ]
@@ -431,7 +431,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "#plot the feature importance for the prediction\n",
+        "# Plot the feature importance for the prediction\n",
         "import numpy as np\n",
         "import matplotlib.pyplot as plt; plt.rcdefaults()\n",
         "\n",

how-to-use-azureml/explain-model/azure-integration/scoring-time/train-explain-model-on-amlcompute-and-deploy.ipynb

@@ -156,7 +156,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Submit an AmlCompute run in a few different ways\n",
+        "## Submit an AmlCompute run\n",
         "\n",
         "First lets check which VM families are available in your region. Azure is a regional service and some specialized SKUs (especially GPUs) are only available in certain regions. Since AmlCompute is created in the region of your workspace, we will use the supported_vms () function to see if the VM family we want to use ('STANDARD_D2_V2') is supported.\n",
         "\n",
@@ -202,9 +202,43 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "### Provision as a run based compute target\n",
+        "### Provision a compute target\n",
         "\n",
-        "You can provision AmlCompute as a compute target at run-time. In this case, the compute is auto-created for your run, scales up to max_nodes that you specify, and then **deleted automatically** after the run completes."
+        "You can provision an AmlCompute resource by simply defining two parameters thanks to smart defaults. By default it autoscales from 0 nodes and provisions dedicated VMs to run your job in a container. This is useful when you want to continously re-use the same target, debug it between jobs or simply share the resource with other users of your workspace.\n",
+        "\n",
+        "* `vm_size`: VM family of the nodes provisioned by AmlCompute. Simply choose from the supported_vmsizes() above\n",
+        "* `max_nodes`: Maximum nodes to autoscale to while running a job on AmlCompute"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "from azureml.core.compute import ComputeTarget, AmlCompute\n",
+        "from azureml.core.compute_target import ComputeTargetException\n",
+        "\n",
+        "# Choose a name for your CPU cluster\n",
+        "cpu_cluster_name = \"cpu-cluster\"\n",
+        "\n",
+        "# Verify that cluster does not exist already\n",
+        "try:\n",
+        "    cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)\n",
+        "    print('Found existing cluster, use it.')\n",
+        "except ComputeTargetException:\n",
+        "    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',\n",
+        "                                                           max_nodes=4)\n",
+        "    cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)\n",
+        "\n",
+        "cpu_cluster.wait_for_completion(show_output=True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "### Configure & Run"
       ]
     },
     {
@@ -217,28 +251,21 @@
         "from azureml.core.conda_dependencies import CondaDependencies\n",
         "from azureml.core.runconfig import DEFAULT_CPU_IMAGE\n",
         "\n",
-        "# create a new runconfig object\n",
+        "# Create a new runconfig object\n",
         "run_config = RunConfiguration()\n",
         "\n",
-        "# signal that you want to use AmlCompute to execute script.\n",
+        "# Set compute target to AmlCompute target created in previous step\n",
-        "run_config.target = \"amlcompute\"\n",
+        "run_config.target = cpu_cluster.name\n",
         "\n",
-        "# AmlCompute will be created in the same region as workspace\n",
+        "# Enable Docker \n",
-        "# Set vm size for AmlCompute\n",
-        "run_config.amlcompute.vm_size = 'STANDARD_D2_V2'\n",
-        "\n",
-        "# enable Docker \n",
         "run_config.environment.docker.enabled = True\n",
         "\n",
-        "# set Docker base image to the default CPU-based image\n",
+        "# Set Docker base image to the default CPU-based image\n",
         "run_config.environment.docker.base_image = DEFAULT_CPU_IMAGE\n",
         "\n",
-        "# use conda_dependencies.yml to create a conda environment in the Docker image for execution\n",
+        "# Use conda_dependencies.yml to create a conda environment in the Docker image for execution\n",
         "run_config.environment.python.user_managed_dependencies = False\n",
         "\n",
-        "# auto-prepare the Docker image when used for execution (if it is not already prepared)\n",
-        "run_config.auto_prepare_environment = True\n",
-        "\n",
         "azureml_pip_packages = [\n",
         "    'azureml-defaults', 'azureml-contrib-interpret', 'azureml-core', 'azureml-telemetry',\n",
         "    'azureml-interpret', 'azureml-dataprep'\n",
@@ -263,7 +290,7 @@
         "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
         "if pandas_ver:\n",
         "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
-        "# specify CondaDependencies obj\n",
+        "# Specify CondaDependencies obj\n",
         "# The CondaDependencies specifies the conda and pip packages that are installed in the environment\n",
         "# the submitted job is run in.  Note the remote environment(s) needs to be similar to the local\n",
         "# environment, otherwise if a model is trained or deployed in a different environment this can\n",
@@ -327,7 +354,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# retrieve model for visualization and deployment\n",
+        "# Retrieve model for visualization and deployment\n",
         "from azureml.core.model import Model\n",
         "import joblib\n",
         "original_model = Model(ws, 'amlcompute_deploy_model')\n",
@@ -341,7 +368,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# retrieve global explanation for visualization\n",
+        "# Retrieve global explanation for visualization\n",
         "from azureml.contrib.interpret.explanation.explanation_client import ExplanationClient\n",
         "\n",
         "# get model explanation data\n",
@@ -355,7 +382,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# retrieve x_test for visualization\n",
+        "# Retrieve x_test for visualization\n",
         "import joblib\n",
         "x_test_path = './x_test.pkl'\n",
         "run.download_file('x_test_ibm.pkl', output_file_path=x_test_path)\n",
@@ -435,7 +462,7 @@
         "    sklearn_dep = 'scikit-learn=={}'.format(sklearn_ver)\n",
         "if pandas_ver:\n",
         "    pandas_dep = 'pandas=={}'.format(pandas_ver)\n",
-        "# specify CondaDependencies obj\n",
+        "# Specify CondaDependencies obj\n",
         "# The CondaDependencies specifies the conda and pip packages that are installed in the environment\n",
         "# the submitted job is run in.  Note the remote environment(s) needs to be similar to the local\n",
         "# environment, otherwise if a model is trained or deployed in a different environment this can\n",
@@ -457,7 +484,7 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "# retrieve scoring explainer for deployment\n",
+        "# Retrieve scoring explainer for deployment\n",
         "scoring_explainer_model = Model(ws, 'IBM_attrition_explainer')"
       ]
     },
@@ -496,17 +523,17 @@
       "source": [
         "import requests\n",
         "\n",
-        "# create data to test service with\n",
+        "# Create data to test service with\n",
         "examples = x_test[:4]\n",
         "input_data = examples.to_json()\n",
         "\n",
         "headers = {'Content-Type':'application/json'}\n",
         "\n",
-        "# send request to service\n",
+        "# Send request to service\n",
         "print(\"POST to url\", service.scoring_uri)\n",
         "resp = requests.post(service.scoring_uri, input_data, headers=headers)\n",
         "\n",
-        "# can covert back to Python objects from json string if desired\n",
+        "# Can covert back to Python objects from json string if desired\n",
         "print(\"prediction:\", resp.text)"
       ]
     },

how-to-use-azureml/reinforcement-learning/cartpole-on-compute-instance/cartpole_ci.ipynb

@@ -22,7 +22,7 @@
       "source": [
         "# Reinforcement Learning in Azure Machine Learning - Cartpole Problem on Compute Instance\n",
         "\n",
-        "Reinforcement Learning in Azure Machine Learning is a managed service for running reinforcement learning training and simulation. With Reinforcement Learning in Azure Machine Learning, data scientists can start developing reinforcement learning systems on one machine, and scale to compute targets with 100\u00e2\u20ac\u2122s of nodes if needed.\n",
+        "Reinforcement Learning in Azure Machine Learning is a managed service for running reinforcement learning training and simulation. With Reinforcement Learning in Azure Machine Learning, data scientists can start developing reinforcement learning systems on one machine, and scale to compute targets with 100s of nodes if needed.\n",
         "\n",
         "This example shows how to use Reinforcement Learning in Azure Machine Learning to train a Cartpole playing agent on a compute instance."
       ]

how-to-use-azureml/reinforcement-learning/cartpole-on-single-compute/cartpole_sc.ipynb

@@ -22,7 +22,7 @@
       "source": [
         "# Reinforcement Learning in Azure Machine Learning - Cartpole Problem on Single Compute\n",
         "\n",
-        "Reinforcement Learning in Azure Machine Learning is a managed service for running reinforcement learning training and simulation. With Reinforcement Learning in Azure Machine Learning, data scientists can start developing reinforcement learning systems on one machine, and scale to compute targets with 100\u00e2\u20ac\u2122s of nodes if needed.\n",
+        "Reinforcement Learning in Azure Machine Learning is a managed service for running reinforcement learning training and simulation. With Reinforcement Learning in Azure Machine Learning, data scientists can start developing reinforcement learning systems on one machine, and scale to compute targets with 100s of nodes if needed.\n",
         "\n",
         "This example shows how to use Reinforcement Learning in Azure Machine Learning to train a Cartpole playing agent on a single compute. "
       ]

how-to-use-azureml/reinforcement-learning/minecraft-on-distributed-compute/docker/cpu/patch_files/ClientStateMachine.java

@@ -1477,7 +1477,7 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
      * Pause the old server. It's vital that we do this, otherwise it will
      * respond to the quit disconnect package straight away and kill the server
      * thread, which means there will be no server to respond to the loadWorld
-     * code. (This was the cause of the infamous "Holder Lookups" hang.)
+     * code.
      */
     public class PauseOldServerEpisode extends ConfigAwareStateEpisode
     {
@@ -1505,7 +1505,6 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
                 {
                     if (!killPublicFlag(Minecraft.getMinecraft().getIntegratedServer()))
                     {
-                        // Can't pause, don't want to risk the hang - so bail.
                         episodeHasCompletedWithErrors(ClientState.ERROR_CANNOT_CREATE_WORLD, "Can not pause the old server since it's open to LAN; no way to safely create new world.");
                     }
                 }
@@ -1555,8 +1554,7 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
             if (inAbortState())
                 episodeHasCompleted(ClientState.MISSION_ABORTED);
 
-            // We need to make sure that both the client and server have paused,
+            // We need to make sure that both the client and server have paused.
-            // otherwise we are still susceptible to the "Holder Lookups" hang.
             
             // Since the server sets its pause state in response to the client's pause state,
             // and it only performs this check once, at the top of its tick method,
@@ -1615,7 +1613,7 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
                 // If the Minecraft server isn't paused at this point,
                 // then the following line will cause the server thread to exit...
                 Minecraft.getMinecraft().world.sendQuittingDisconnectingPacket();
-                // ...in which case the next line will hang.
+                // ...in which case the next line will block.
                 Minecraft.getMinecraft().loadWorld((WorldClient) null);
                 // Must display the GUI or Minecraft will attempt to access a non-existent player in the client tick.
                 Minecraft.getMinecraft().displayGuiScreen(new GuiMainMenu());
@@ -2135,7 +2133,6 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
                         envServer.observation(data);
                     }
                 } else {
-                    // Bung the whole shebang off via TCP:
                     if (this.observationSocket.sendTCPString(data)) {
                         this.failedTCPObservationSendCount = 0;
                     } else {

how-to-use-azureml/reinforcement-learning/minecraft-on-distributed-compute/docker/gpu/patch_files/ClientStateMachine.java

@@ -1477,7 +1477,7 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
      * Pause the old server. It's vital that we do this, otherwise it will
      * respond to the quit disconnect package straight away and kill the server
      * thread, which means there will be no server to respond to the loadWorld
-     * code. (This was the cause of the infamous "Holder Lookups" hang.)
+     * code.
      */
     public class PauseOldServerEpisode extends ConfigAwareStateEpisode
     {
@@ -1505,7 +1505,6 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
                 {
                     if (!killPublicFlag(Minecraft.getMinecraft().getIntegratedServer()))
                     {
-                        // Can't pause, don't want to risk the hang - so bail.
                         episodeHasCompletedWithErrors(ClientState.ERROR_CANNOT_CREATE_WORLD, "Can not pause the old server since it's open to LAN; no way to safely create new world.");
                     }
                 }
@@ -1555,8 +1554,7 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
             if (inAbortState())
                 episodeHasCompleted(ClientState.MISSION_ABORTED);
 
-            // We need to make sure that both the client and server have paused,
+            // We need to make sure that both the client and server have paused.
-            // otherwise we are still susceptible to the "Holder Lookups" hang.
             
             // Since the server sets its pause state in response to the client's pause state,
             // and it only performs this check once, at the top of its tick method,
@@ -1615,7 +1613,7 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
                 // If the Minecraft server isn't paused at this point,
                 // then the following line will cause the server thread to exit...
                 Minecraft.getMinecraft().world.sendQuittingDisconnectingPacket();
-                // ...in which case the next line will hang.
+                // ...in which case the next line will block.
                 Minecraft.getMinecraft().loadWorld((WorldClient) null);
                 // Must display the GUI or Minecraft will attempt to access a non-existent player in the client tick.
                 Minecraft.getMinecraft().displayGuiScreen(new GuiMainMenu());
@@ -2135,7 +2133,6 @@ public class ClientStateMachine extends StateMachine implements IMalmoMessageLis
                         envServer.observation(data);
                     }
                 } else {
-                    // Bung the whole shebang off via TCP:
                     if (this.observationSocket.sendTCPString(data)) {
                         this.failedTCPObservationSendCount = 0;
                     } else {

how-to-use-azureml/reinforcement-learning/setup/devenv_setup.yml

@@ -2,4 +2,3 @@ name: devenv_setup
 dependencies:
 - pip:
   - azureml-sdk
-  - azure-mgmt-network

how-to-use-azureml/track-and-monitor-experiments/logging-api/logging-api.ipynb

@@ -100,7 +100,7 @@
         "\n",
         "# Check core SDK version number\n",
         "\n",
-        "print(\"This notebook was created using SDK version 1.9.0, you are currently running version\", azureml.core.VERSION)"
+        "print(\"This notebook was created using SDK version 1.10.0, you are currently running version\", azureml.core.VERSION)"
       ]
     },
     {

how-to-use-azureml/track-and-monitor-experiments/manage-runs/manage-runs.ipynb

@@ -215,6 +215,24 @@
         "!more hello.py"
       ]
     },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Submitted runs take a snapshot of the *source_directory* to use when executing. You can control which files are available to the run by using an *.amlignore* file."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "%%writefile .amlignore\n",
+        "# Exclude the outputs directory automatically created by our earlier runs.\n",
+        "/outputs"
+      ]
+    },
     {
       "cell_type": "markdown",
       "metadata": {},
@@ -373,7 +391,7 @@
       "source": [
         "## Query properties and tags\n",
         "\n",
-        "You can quary runs within an experiment that match specific properties and tags. "
+        "You can query runs within an experiment that match specific properties and tags."
       ]
     },
     {
@@ -579,6 +597,22 @@
         "name": "roastala"
       }
     ],
+    "category": "training",
+    "compute": [
+      "Local"
+    ],
+    "datasets": [
+      "None"
+    ],
+    "deployment": [
+      "None"
+    ],
+    "exclude_from_index": false,
+    "framework": [
+      "None"
+    ],
+    "friendly_name": "Managing your training runs",
+    "index_order": 2,
     "kernelspec": {
       "display_name": "Python 3.6",
       "language": "python",
@@ -596,26 +630,10 @@
       "pygments_lexer": "ipython3",
       "version": "3.6.5"
     },
-    "friendly_name": "Managing your training runs",
-    "exclude_from_index": false,
-    "index_order": 2,
-    "category": "training",
-    "task": "Monitor and complete runs",
-    "datasets": [
-      "None"
-    ],
-    "compute": [
-      "Local"
-    ],
-    "deployment": [
-      "None"
-    ],
-    "framework": [
-      "None"
-    ],
     "tags": [
       "None"
-    ]
+    ],
+    "task": "Monitor and complete runs"
   },
   "nbformat": 4,
   "nbformat_minor": 2

how-to-use-azureml/training/train-on-amlcompute/train-on-amlcompute.ipynb

@@ -184,7 +184,7 @@
         "myenv = Environment(\"myenv\")\n",
         "\n",
         "myenv.docker.enabled = True\n",
-        "myenv.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
+        "myenv.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn', 'packaging'])"
       ]
     },
     {

how-to-use-azureml/training/train-on-computeinstance/train-on-computeinstance.ipynb

@@ -129,7 +129,8 @@
         "from azureml.core.compute_target import ComputeTargetException\n",
         "\n",
         "# Choose a name for your instance\n",
-        "compute_name = \"compute-instance\"\n",
+        "# Compute instance name should be unique across the azure region\n",
+        "compute_name = \"ci{}\".format(ws._workspace_id)[:10]\n",
         "\n",
         "# Verify that instance does not exist already\n",
         "try:\n",
@@ -182,7 +183,7 @@
       "source": [
         "### Create environment\n",
         "\n",
-        "Create Docker based environment with scikit-learn installed."
+        "Create an environment with scikit-learn installed."
       ]
     },
     {
@@ -195,8 +196,6 @@
         "from azureml.core.conda_dependencies import CondaDependencies\n",
         "\n",
         "myenv = Environment(\"myenv\")\n",
-        "\n",
-        "myenv.docker.enabled = True\n",
         "myenv.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
       ]
     },
@@ -240,9 +239,15 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "%%time\n",
+        "from azureml.widgets import RunDetails\n",
-        "# Shows output of the run on stdout.\n",
+        "RunDetails(run).show()"
-        "run.wait_for_completion(show_output=True)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "You can use the get_active_runs() to get the currently running or queued jobs on the compute instance"
       ]
     },
     {
@@ -251,6 +256,33 @@
       "metadata": {},
       "outputs": [],
       "source": [
+        "# wait for the run to reach Queued or Running state if it is in Preparing state\n",
+        "status = run.get_status()\n",
+        "while status not in ['Queued', 'Running', 'Completed', 'Failed', 'Canceled']:\n",
+        "   state = run.get_status()\n",
+        "   print('Run status: {}'.format(status))\n",
+        "   time.sleep(10)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# get active runs which are in Queued or Running state\n",
+        "active_runs = instance.get_active_runs()\n",
+        "for active_run in active_runs:\n",
+        "    print(active_run.run_id, ',', active_run.status)"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "run.wait_for_completion()\n",
         "print(run.get_metrics())"
       ]
     },
@@ -321,7 +353,7 @@
       "outputs": [],
       "source": [
         "# delete() is used to delete the ComputeInstance target. Useful if you want to re-use the compute name \n",
-        "# instance.delete()"
+        "# instance.delete(wait_for_completion=True, show_output=True)"
       ]
     }
   ],

setup-environment/configuration.ipynb

@@ -102,7 +102,7 @@
       "source": [
         "import azureml.core\n",
         "\n",
-        "print(\"This notebook was created using version 1.9.0 of the Azure ML SDK\")\n",
+        "print(\"This notebook was created using version 1.10.0 of the Azure ML SDK\")\n",
         "print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
       ]
     },

tutorials/regression-automl-nyc-taxi-data/regression-automated-ml.ipynb

@@ -269,10 +269,7 @@
       "source": [
         "from sklearn.model_selection import train_test_split\n",
         "\n",
-        "y_df = final_df.pop(\"totalAmount\")\n",
+        "x_train, x_test = train_test_split(final_df, test_size=0.2, random_state=223)"
-        "x_df = final_df\n",
-        "\n",
-        "x_train, x_test, y_train, y_test = train_test_split(x_df, y_df, test_size=0.2, random_state=223)"
       ]
     },
     {
@@ -311,15 +308,16 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Define the experiment parameter and model settings for training. View the full list of [settings](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-auto-train). Submitting the experiment with these default settings will take approximately 5-10 min, but if you want a shorter run time, reduce the `iterations` parameter.\n",
+        "Define the experiment parameter and model settings for training. View the full list of [settings](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-auto-train). Submitting the experiment with these default settings will take approximately 20 minutes, but if you want a shorter run time, reduce the `experiment_timeout_hours` parameter.\n",
         "\n",
         "\n",
         "|Property| Value in this tutorial |Description|\n",
         "|----|----|---|\n",
         "|**iteration_timeout_minutes**|2|Time limit in minutes for each iteration. Reduce this value to decrease total runtime.|\n",
-        "|**iterations**|20|Number of iterations. In each iteration, a new machine learning model is trained with your data. This is the primary value that affects total run time.|\n",
+        "|**experiment_timeout_hours**|0.3|Maximum amount of time in hours that all iterations combined can take before the experiment terminates.|\n",
+        "|**enable_early_stopping**|True|Flag to enable early termination if the score is not improving in the short term.|\n",
         "|**primary_metric**| spearman_correlation | Metric that you want to optimize. The best-fit model will be chosen based on this metric.|\n",
-        "|**preprocess**| True | By using **True**, the experiment can preprocess the input data (handling missing data, converting text to numeric, etc.)|\n",
+        "|**featurization**| auto | By using auto, the experiment can preprocess the input data (handling missing data, converting text to numeric, etc.)|\n",
         "|**verbosity**| logging.INFO | Controls the level of logging.|\n",
         "|**n_cross_validations**|5|Number of cross-validation splits to perform when validation data is not specified.|"
       ]
@@ -334,9 +332,10 @@
         "\n",
         "automl_settings = {\n",
         "    \"iteration_timeout_minutes\": 2,\n",
-        "    \"iterations\": 20,\n",
+        "    \"experiment_timeout_hours\": 0.3,\n",
+        "    \"enable_early_stopping\": True,\n",
         "    \"primary_metric\": 'spearman_correlation',\n",
-        "    \"preprocess\": True,\n",
+        "    \"featurization\": 'auto',\n",
         "    \"verbosity\": logging.INFO,\n",
         "    \"n_cross_validations\": 5\n",
         "}"
@@ -359,8 +358,8 @@
         "\n",
         "automl_config = AutoMLConfig(task='regression',\n",
         "                             debug_log='automated_ml_errors.log',\n",
-        "                             X=x_train.values,\n",
+        "                             training_data=x_train,\n",
-        "                             y=y_train.values.flatten(),\n",
+        "                             label_column_name=\"totalAmount\",\n",
         "                             **automl_settings)"
       ]
     },
@@ -467,7 +466,9 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "y_predict = fitted_model.predict(x_test.values)\n",
+        "y_test = x_test.pop(\"totalAmount\")\n",
+        "\n",
+        "y_predict = fitted_model.predict(x_test)\n",
         "print(y_predict[:10])"
       ]
     },