jprdonnelly/MachineLearningNotebooks
1
0
Fork 0
mirror of https://github.com/Azure/MachineLearningNotebooks.git synced 2025-12-19 17:17:04 -05:00
Code Issues Projects Releases Wiki Activity

update samples from Release-130 as a part of SDK release

Browse Source
This commit is contained in:
amlrelsa-ms
2022-03-29 22:33:38 +00:00
parent 796798cb49
commit 95b0392ed2
534 changed files with 151904 additions and 27048 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
tutorials/README.md Normal file
View File

@@ -0,0 +1,39 @@
# Azure Machine Learning Tutorials
Azure Machine Learning, a cloud-based environment you can use to train, deploy, automate, manage, and track ML models.
Azure Machine Learning can be used for any kind of machine learning, from classical ML to supervised, unsupervised, and deep learning.
This folder contains a collection of Jupyter Notebooks with the code used in accompanying step-by-step tutorials.
## Set up your environment.
If you are using an Azure Machine Learning Notebook VM, everything is already set up for you. Otherwise, see the [get started creating your first ML experiment with the Python SDK tutorial](https://docs.microsoft.com/en-us/azure/machine-learning/tutorial-1st-experiment-sdk-setup).
## Introductory Samples
The following tutorials are intended to provide an introductory overview of Azure Machine Learning.
| Tutorial | Description | Notebook | Task | Framework |
| --- | --- | --- | --- | --- |
| [Train your first ML Model](https://docs.microsoft.com/azure/machine-learning/tutorial-1st-experiment-sdk-train) | Learn the foundational design patterns in Azure Machine Learning and train a scikit-learn model based on a diabetes data set. | [tutorial-quickstart-train-model.ipynb](create-first-ml-experiment/tutorial-1st-experiment-sdk-train.ipynb) | Regression | Scikit-Learn
| [Train an image classification model](https://docs.microsoft.com/azure/machine-learning/tutorial-train-models-with-aml) | Train a scikit-learn image classification model. | [img-classification-part1-training.ipynb](image-classification-mnist-data/img-classification-part1-training.ipynb) | Image Classification | Scikit-Learn
| [Deploy an image classification model](https://docs.microsoft.com/azure/machine-learning/tutorial-deploy-models-with-aml) | Deploy a scikit-learn image classification model to Azure Container Instances. | [img-classification-part2-deploy.ipynb](image-classification-mnist-data/img-classification-part2-deploy.ipynb) | Image Classification | Scikit-Learn
| [Deploy an encrypted inferencing service](https://docs.microsoft.com/azure/machine-learning/tutorial-deploy-models-with-aml) |Deploy an image classification model for encrypted inferencing in Azure Container Instances | [img-classification-part3-deploy-encrypted.ipynb](image-classification-mnist-data/img-classification-part3-deploy-encrypted.ipynb) | Image Classification | Scikit-Learn
| [Use automated machine learning to predict taxi fares](https://docs.microsoft.com/azure/machine-learning/tutorial-auto-train-models) | Train a regression model to predict taxi fares using Automated Machine Learning. | [regression-part2-automated-ml.ipynb](regression-automl-nyc-taxi-data/regression-automated-ml.ipynb) | Regression | Automated ML
| Azure ML in 10 minutes (Compute instance required) |Learn how to run an image classification model, track model metrics, and deploy a model in 10 minutes. | [quickstart-azureml-in-10mins.ipynb](compute-instance-quickstarts/quickstart-azureml-in-10mins/quickstart-azureml-in-10mins.ipynb) | Image Classification | Scikit-Learn |
| Get started with Azure ML Job Submission (Compute instance required) |Learn how to use the Azure Machine Learning Python SDK to submit batch jobs. | [quickstart-azureml-python-sdk.ipynb](compute-instance-quickstarts/quickstart-azureml-python-sdk/quickstart-azureml-python-sdk.ipynb) | Image Classification | Scikit-Learn |
| Get started with Automated ML (Compute instance required) | Learn how to use Automated ML for Fraud classification. | [quickstart-azureml-automl.ipynb](compute-instance-quickstarts/quickstart-azureml-automl/quickstart-azureml-automl.ipynb) | Classification | Automated ML |
## Advanced Samples
The following tutorials are intended to provide examples of more advanced feature in Azure Machine Learning.
| Tutorial | Description | Notebook | Task | Framework |
| --- | --- | --- | --- | --- |
| [Build an Azure Machine Learning pipeline for batch scoring](https://docs.microsoft.com/azure/machine-learning/tutorial-pipeline-batch-scoring-classification) | Create an Azure Machine Learning pipeline to run batch scoring image classification jobs | [tutorial-pipeline-batch-scoring-classification.ipynb](machine-learning-pipelines-advanced/tutorial-pipeline-batch-scoring-classification.ipynb) | Image Classification | TensorFlow
For additional documentation and resources, see the [official documentation site for Azure Machine Learning](https://docs.microsoft.com/azure/machine-learning/).
![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/README.png)

498
tutorials/compute-instance-quickstarts/quickstart-azureml-automl/quickstart-azureml-automl.ipynb Normal file
View File

@@ -0,0 +1,498 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/quickstart-ci/ClassificationWithAutomatedML.png)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"# Quickstart: Fraud Classification using Automated ML\n",
"\n",
"In this quickstart, you use automated machine learning in Azure Machine Learning service to train a classification model on an associated fraud credit card dataset. This process accepts training data and configuration settings, and automatically iterates through combinations of different feature normalization/standardization methods, models, and hyperparameter settings to arrive at the best model.\n",
"\n",
"You will learn how to:\n",
"\n",
"> * Download a dataset and look at the data\n",
"> * Train a machine learning classification model using autoML \n",
"> * Explore the results\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Connect to your workspace and create an experiment\n",
"\n",
"You start with importing some libraries and creating an experiment to track the runs in your workspace. A workspace can have multiple experiments, and all the users that have access to the workspace can collaborate on them. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612968646250
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"import logging\n",
"\n",
"from matplotlib import pyplot as plt\n",
"import pandas as pd\n",
"import numpy as np\n",
"\n",
"import azureml.core\n",
"from azureml.core.experiment import Experiment\n",
"from azureml.core.workspace import Workspace\n",
"from azureml.core.dataset import Dataset\n",
"from azureml.train.automl import AutoMLConfig"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612968706273
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"ws = Workspace.from_config()\n",
"\n",
"# choose a name for your experiment\n",
"experiment_name = \"fraud-classification-automl-tutorial\"\n",
"\n",
"experiment = Experiment(ws, experiment_name)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Load Data\n",
"\n",
"Load the credit card dataset from a csv file containing both training features and labels. The features are inputs to the model, while the training labels represent the expected output of the model. Next, we'll split the data using random_split and extract the training data for the model.\n",
"\n",
"\n",
"Follow this [how-to](https://aka.ms/azureml/howto/createdatasets) if you want to learn more about Datasets and how to use them.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612968722555
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"data = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/creditcard.csv\"\n",
"dataset = Dataset.Tabular.from_delimited_files(data)\n",
"training_data, validation_data = dataset.random_split(percentage=0.8, seed=223)\n",
"label_column_name = \"Class\""
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Train\n",
"\n",
"\n",
"\n",
"When you use automated machine learning in Azure ML, you input training data and configuration settings, and the process automatically iterates through combinations of different feature normalization/standardization methods, models, and hyperparameter settings to arrive at the best model. \n",
"Learn more about how you configure automated ML [here](https://docs.microsoft.com/azure/machine-learning/how-to-configure-auto-train).\n",
"\n",
"\n",
"Instantiate an [AutoMLConfig](https://docs.microsoft.com/python/api/azureml-train-automl-client/azureml.train.automl.automlconfig.automlconfig?view=azure-ml-py) object. This defines the settings and data used to run the experiment.\n",
"\n",
"|Property|Description|\n",
"|-|-|\n",
"|**task**|classification or regression|\n",
"|**primary_metric**|This is the metric that you want to optimize. \n",
"|**enable_early_stopping** | Stop the run if the metric score is not showing improvement.|\n",
"|**n_cross_validations**|Number of cross validation splits.|\n",
"|**training_data**|Input dataset, containing both features and label column.|\n",
"|**label_column_name**|The name of the label column.|\n",
"\n",
"You can find more information about primary metrics [here](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-auto-train#primary-metric)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612968806233
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"automl_settings = {\n",
" \"n_cross_validations\": 3,\n",
" \"primary_metric\": \"average_precision_score_weighted\",\n",
" \"experiment_timeout_hours\": 0.25, # This is a time limit for testing purposes, remove it for real use cases, this will drastically limit ability to find the best model possible\n",
" \"verbosity\": logging.INFO,\n",
" \"enable_stack_ensemble\": False,\n",
"}\n",
"\n",
"automl_config = AutoMLConfig(\n",
" task=\"classification\",\n",
" debug_log=\"automl_errors.log\",\n",
" training_data=training_data,\n",
" label_column_name=label_column_name,\n",
" **automl_settings,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"Call the `submit` method on the experiment object and pass the run configuration. \n",
"\n",
"**Note: Depending on the data and the number of iterations an AutoML run can take a while to complete.**\n",
"\n",
"In this example, we specify `show_output = True` to print currently running iterations to the console. It is also possible to navigate to the experiment through the **Experiment** activity tab in the left menu, and monitor the run status from there."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612970125369
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"local_run = experiment.submit(automl_config, show_output=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612976292559
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"local_run"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Analyze results\n",
"\n",
"Below we select the best model from our iterations. The `get_output` method on `automl_classifier` returns the best run and the model for the run."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612976298373
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"best_run, best_model = local_run.get_output()\n",
"best_model"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Tests\n",
"\n",
"Now that the model is trained, split the data in the same way the data was split for training (The difference here is the data is being split locally) and then run the test data through the trained model to get the predicted values."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612976320370
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# convert the test data to dataframe\n",
"X_test_df = validation_data.drop_columns(\n",
" columns=[label_column_name]\n",
").to_pandas_dataframe()\n",
"y_test_df = validation_data.keep_columns(\n",
" columns=[label_column_name], validate=True\n",
").to_pandas_dataframe()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612976325829
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# call the predict functions on the model\n",
"y_pred = best_model.predict(X_test_df)\n",
"y_pred"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"\n",
"\n",
"### Calculate metrics for the prediction\n",
"\n",
"Now visualize the data to show what our truth (actual) values are compared to the predicted values \n",
"from the trained model that was returned.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612976330108
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"from sklearn.metrics import confusion_matrix\n",
"import numpy as np\n",
"import itertools\n",
"\n",
"cf = confusion_matrix(y_test_df.values, y_pred)\n",
"plt.imshow(cf, cmap=plt.cm.Blues, interpolation=\"nearest\")\n",
"plt.colorbar()\n",
"plt.title(\"Confusion Matrix\")\n",
"plt.xlabel(\"Predicted\")\n",
"plt.ylabel(\"Actual\")\n",
"class_labels = [\"False\", \"True\"]\n",
"tick_marks = np.arange(len(class_labels))\n",
"plt.xticks(tick_marks, class_labels)\n",
"plt.yticks([-0.5, 0, 1, 1.5], [\"\", \"False\", \"True\", \"\"])\n",
"# plotting text value inside cells\n",
"thresh = cf.max() / 2.0\n",
"for i, j in itertools.product(range(cf.shape[0]), range(cf.shape[1])):\n",
" plt.text(\n",
" j,\n",
" i,\n",
" format(cf[i, j], \"d\"),\n",
" horizontalalignment=\"center\",\n",
" color=\"white\" if cf[i, j] > thresh else \"black\",\n",
" )\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Control cost and further exploration\n",
"\n",
"If you want to control cost you can stop the compute instance this notebook is running on by clicking the \"Stop compute\" button next to the status dropdown in the menu above.\n",
"\n",
"\n",
"If you want to run more notebook samples, you can click on **Sample Notebooks** next to the **Files** view and explore the notebooks made available for you there."
]
}
],
"metadata": {
"authors": [
{
"name": "cewidste"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
},
"notice": "Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License.",
"nteract": {
"version": "nteract-front-end@1.0.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

4
tutorials/compute-instance-quickstarts/quickstart-azureml-automl/quickstart-azureml-automl.yml Normal file
View File

@@ -0,0 +1,4 @@
name: quickstart-azureml-automl
dependencies:
- pip:
- azureml-sdk

502
tutorials/compute-instance-quickstarts/quickstart-azureml-in-10mins/quickstart-azureml-in-10mins.ipynb Normal file
View File

@@ -0,0 +1,502 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/quickstart-ci/AzureMLin10mins.png)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"# Quickstart: Train and deploy a model in Azure Machine Learning in 10 minutes\n",
"\n",
"In this quickstart, learn how to get started with Azure Machine Learning. You'll train an image classification model using the [MNIST](https://docs.microsoft.com/azure/open-datasets/dataset-mnist) dataset.\n",
"\n",
"You'll learn how to:\n",
"\n",
"* Download a dataset and look at the data\n",
"* Train an image classification model and log metrics using MLflow\n",
"* Deploy the model to do real-time inference"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Import Data\n",
"\n",
"Before you train a model, you need to understand the data you're using to train it. In this section, learn how to:\n",
"\n",
"* Download the MNIST dataset\n",
"* Display some sample images\n",
"\n",
"You'll use Azure Open Datasets to get the raw MNIST data files. [Azure Open Datasets](https://docs.microsoft.com/azure/open-datasets/overview-what-are-open-datasets) are curated public datasets that you can use to add scenario-specific features to machine learning solutions for better models. Each dataset has a corresponding class, `MNIST` in this case, to retrieve the data in different ways."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from azureml.opendatasets import MNIST\n",
"\n",
"data_folder = os.path.join(os.getcwd(), \"/tmp/qs_data\")\n",
"os.makedirs(data_folder, exist_ok=True)\n",
"\n",
"mnist_file_dataset = MNIST.get_file_dataset()\n",
"mnist_file_dataset.download(data_folder, overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Take a look at the data\n",
"\n",
"Load the compressed files into `numpy` arrays. Then use `matplotlib` to plot 30 random images from the dataset with their labels above them. \n",
"\n",
"Note this step requires a `load_data` function that's included in an `utils.py` file. This file is placed in the same folder as this notebook. The `load_data` function simply parses the compressed files into numpy arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from utils import load_data\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import glob\n",
"\n",
"\n",
"# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the model converge faster.\n",
"X_train = (\n",
" load_data(\n",
" glob.glob(\n",
" os.path.join(data_folder, \"**/train-images-idx3-ubyte.gz\"), recursive=True\n",
" )[0],\n",
" False,\n",
" )\n",
" / 255.0\n",
")\n",
"X_test = (\n",
" load_data(\n",
" glob.glob(\n",
" os.path.join(data_folder, \"**/t10k-images-idx3-ubyte.gz\"), recursive=True\n",
" )[0],\n",
" False,\n",
" )\n",
" / 255.0\n",
")\n",
"y_train = load_data(\n",
" glob.glob(\n",
" os.path.join(data_folder, \"**/train-labels-idx1-ubyte.gz\"), recursive=True\n",
" )[0],\n",
" True,\n",
").reshape(-1)\n",
"y_test = load_data(\n",
" glob.glob(\n",
" os.path.join(data_folder, \"**/t10k-labels-idx1-ubyte.gz\"), recursive=True\n",
" )[0],\n",
" True,\n",
").reshape(-1)\n",
"\n",
"\n",
"# now let's show some randomly chosen images from the traininng set.\n",
"count = 0\n",
"sample_size = 30\n",
"plt.figure(figsize=(16, 6))\n",
"for i in np.random.permutation(X_train.shape[0])[:sample_size]:\n",
" count = count + 1\n",
" plt.subplot(1, sample_size, count)\n",
" plt.axhline(\"\")\n",
" plt.axvline(\"\")\n",
" plt.text(x=10, y=-10, s=y_train[i], fontsize=18)\n",
" plt.imshow(X_train[i].reshape(28, 28), cmap=plt.cm.Greys)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Train model and log metrics with MLflow\n",
"\n",
"You'll train the model using the code below. Note that you are using MLflow autologging to track metrics and log model artefacts.\n",
"\n",
"You'll be using the [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) classifier from the [SciKit Learn framework](https://scikit-learn.org/) to classify the data.\n",
"\n",
"**Note: The model training takes approximately 2 minutes to complete.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612966046970
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# create the model\n",
"import mlflow\n",
"import numpy as np\n",
"from sklearn.linear_model import LogisticRegression\n",
"from azureml.core import Workspace\n",
"\n",
"# connect to your workspace\n",
"ws = Workspace.from_config()\n",
"\n",
"# create experiment and start logging to a new run in the experiment\n",
"experiment_name = \"azure-ml-in10-mins-tutorial\"\n",
"\n",
"# set up MLflow to track the metrics\n",
"mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())\n",
"mlflow.set_experiment(experiment_name)\n",
"mlflow.autolog()\n",
"\n",
"# set up the Logistic regression model\n",
"reg = 0.5\n",
"clf = LogisticRegression(\n",
" C=1.0 / reg, solver=\"liblinear\", multi_class=\"auto\", random_state=42\n",
")\n",
"\n",
"# train the model\n",
"with mlflow.start_run() as run:\n",
" clf.fit(X_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## View Experiment\n",
"In the left-hand menu in Azure Machine Learning Studio, select __Experiments__ and then select your experiment (azure-ml-in10-mins-tutorial). An experiment is a grouping of many runs from a specified script or piece of code. Information for the run is stored under that experiment. If the name doesn't exist when you submit an experiment, if you select your run you will see various tabs containing metrics, logs, explanations, etc.\n",
"\n",
"## Version control your models with the model registry\n",
"\n",
"You can use model registration to store and version your models in your workspace. Registered models are identified by name and version. Each time you register a model with the same name as an existing one, the registry increments the version. The code below registers and versions the model you trained above. Once you have executed the code cell below you will be able to see the model in the registry by selecting __Models__ in the left-hand menu in Azure Machine Learning Studio."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612881042710
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# register the model\n",
"model_uri = \"runs:/{}/model\".format(run.info.run_id)\n",
"model = mlflow.register_model(model_uri, \"sklearn_mnist_model\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy the model for real-time inference\n",
"In this section you learn how to deploy a model so that an application can consume (inference) the model over REST.\n",
"\n",
"### Create deployment configuration\n",
"The code cell gets a _curated environment_, which specifies all the dependencies required to host the model (for example, the packages like scikit-learn). Also, you create a _deployment configuration_, which specifies the amount of compute required to host the model. In this case, the compute will have 1CPU and 1GB memory."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612881061728
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# create environment for the deploy\n",
"from azureml.core.environment import Environment\n",
"from azureml.core.conda_dependencies import CondaDependencies\n",
"from azureml.core.webservice import AciWebservice\n",
"\n",
"# get a curated environment\n",
"env = Environment.get(\n",
" workspace=ws, \n",
" name=\"AzureML-sklearn-1.0-ubuntu20.04-py38-cpu\",\n",
" version=1\n",
")\n",
"env.inferencing_stack_version='latest'\n",
"\n",
"# create deployment config i.e. compute resources\n",
"aciconfig = AciWebservice.deploy_configuration(\n",
" cpu_cores=1,\n",
" memory_gb=1,\n",
" tags={\"data\": \"MNIST\", \"method\": \"sklearn\"},\n",
" description=\"Predict MNIST with sklearn\",\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Deploy model\n",
"\n",
"This next code cell deploys the model to Azure Container Instance (ACI).\n",
"\n",
"**Note: The deployment takes approximately 3 minutes to complete.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"%%time\n",
"import uuid\n",
"from azureml.core.model import InferenceConfig\n",
"from azureml.core.environment import Environment\n",
"from azureml.core.model import Model\n",
"\n",
"# get the registered model\n",
"model = Model(ws, \"sklearn_mnist_model\")\n",
"\n",
"# create an inference config i.e. the scoring script and environment\n",
"inference_config = InferenceConfig(entry_script=\"score.py\", environment=env)\n",
"\n",
"# deploy the service\n",
"service_name = \"sklearn-mnist-svc-\" + str(uuid.uuid4())[:4]\n",
"service = Model.deploy(\n",
" workspace=ws,\n",
" name=service_name,\n",
" models=[model],\n",
" inference_config=inference_config,\n",
" deployment_config=aciconfig,\n",
")\n",
"\n",
"service.wait_for_deployment(show_output=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The [*scoring script*](score.py) file referenced in the code above can be found in the same folder as this notebook, and has two functions:\n",
"\n",
"1. an `init` function that executes once when the service starts - in this function you normally get the model from the registry and set global variables\n",
"1. a `run(data)` function that executes each time a call is made to the service. In this function, you normally format the input data, run a prediction, and output the predicted result.\n",
"\n",
"### View Endpoint\n",
"Once the model has been successfully deployed, you can view the endpoint by navigating to __Endpoints__ in the left-hand menu in Azure Machine Learning Studio. You will be able to see the state of the endpoint (healthy/unhealthy), logs, and consume (how applications can consume the model)."
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Test the model service\n",
"\n",
"You can test the model by sending a raw HTTP request to test the web service. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612881538381
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# send raw HTTP request to test the web service.\n",
"import requests\n",
"\n",
"# send a random row from the test set to score\n",
"random_index = np.random.randint(0, len(X_test) - 1)\n",
"input_data = '{\"data\": [' + str(list(X_test[random_index])) + \"]}\"\n",
"\n",
"headers = {\"Content-Type\": \"application/json\"}\n",
"\n",
"resp = requests.post(service.scoring_uri, input_data, headers=headers)\n",
"\n",
"print(\"POST to url\", service.scoring_uri)\n",
"print(\"label:\", y_test[random_index])\n",
"print(\"prediction:\", resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up resources\n",
"\n",
"If you're not going to continue to use this model, delete the Model service using:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612881556520
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# if you want to keep workspace and only delete endpoint (it will incur cost while running)\n",
"service.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you want to control cost further, stop the compute instance by selecting the \"Stop compute\" button next to the **Compute** dropdown. Then start the compute instance again the next time you need it."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Next Steps\n",
"\n",
"In this quickstart, you learned how to run machine learning code in Azure Machine Learning.\n",
"\n",
"Now that you have working code in a development environment, learn how to submit a **_job_** - ideally on a schedule or trigger (for example, arrival of new data).\n",
"\n",
" [**Learn how to get started with Azure ML Job Submission**](../quickstart-azureml-python-sdk/quickstart-azureml-python-sdk.ipynb) "
]
}
],
"metadata": {
"authors": [
{
"name": "cewidste"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
},
"notice": "Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License.",
"nteract": {
"version": "nteract-front-end@1.0.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

11
tutorials/compute-instance-quickstarts/quickstart-azureml-in-10mins/quickstart-azureml-in-10mins.yml Normal file
View File

@@ -0,0 +1,11 @@
name: quickstart-azureml-in-10mins
dependencies:
- pip:
- azureml-sdk
- sklearn
- numpy
- matplotlib
- joblib
- uuid
- requests
- azureml-opendatasets

21
tutorials/compute-instance-quickstarts/quickstart-azureml-in-10mins/score.py Normal file
View File

@@ -0,0 +1,21 @@
import json
import numpy as np
import os
import joblib
def init():
global model
# AZUREML_MODEL_DIR is an environment variable created during deployment.
# It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)
# For multiple models, it points to the folder containing all deployed models (./azureml-models)
model_path = os.path.join(os.getenv("AZUREML_MODEL_DIR"), "model/model.pkl")
model = joblib.load(model_path)
def run(raw_data):
data = np.array(json.loads(raw_data)["data"])
# make prediction
y_hat = model.predict(data)
# you can return any data type as long as it is JSON-serializable
return y_hat.tolist()

24
tutorials/compute-instance-quickstarts/quickstart-azureml-in-10mins/utils.py Normal file
View File

@@ -0,0 +1,24 @@
import gzip
import numpy as np
import struct
# load compressed MNIST gz files and return numpy arrays
def load_data(filename, label=False):
with gzip.open(filename) as gz:
struct.unpack("I", gz.read(4))
n_items = struct.unpack(">I", gz.read(4))
if not label:
n_rows = struct.unpack(">I", gz.read(4))[0]
n_cols = struct.unpack(">I", gz.read(4))[0]
res = np.frombuffer(gz.read(n_items[0] * n_rows * n_cols), dtype=np.uint8)
res = res.reshape(n_items[0], n_rows * n_cols)
else:
res = np.frombuffer(gz.read(n_items[0]), dtype=np.uint8)
res = res.reshape(n_items[0], 1)
return res
# one-hot encode a 1-D array
def one_hot_encode(array, num_of_classes):
return np.eye(num_of_classes)[array.reshape(-1)]

355
tutorials/compute-instance-quickstarts/quickstart-azureml-python-sdk/quickstart-azureml-python-sdk.ipynb Normal file
View File

@@ -0,0 +1,355 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/quickstart-ci/GettingStartedWithPythonSDK.png)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"# Quickstart: Learn how to submit batch jobs with the Azure Machine Learning Python SDK\n",
"\n",
"In this quickstart, you learn how to submit a batch training job using the Python SDK. In this example, we submit the job to the 'local' machine (the compute instance you are running this notebook on). However, you can use exactly the same method to submit the job to different compute targets (for example, AKS, Azure Machine Learning Compute Cluster, Synapse, etc) by changing a single line of code. A full list of support compute targets can be viewed [here](https://docs.microsoft.com/en-us/azure/machine-learning/concept-compute-target). \n",
"\n",
"This quickstart trains a simple logistic regression using the [MNIST](https://docs.microsoft.com/azure/open-datasets/dataset-mnist) dataset and [scikit-learn](http://scikit-learn.org) with Azure Machine Learning. MNIST is a popular dataset consisting of 70,000 grayscale images. Each image is a handwritten digit of 28x28 pixels, representing a number from 0 to 9. The goal is to create a multi-class classifier to identify the digit a given image represents. \n",
"\n",
"You will learn how to:\n",
"\n",
"> * Download a dataset and look at the data\n",
"> * Train an image classification model by submitting a batch job to a compute resource\n",
"> * Use MLflow autologging to track model metrics and log the model artefact\n",
"> * Review training results, find and register the best model"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Connect to your workspace and create an experiment\n",
"\n",
"You start with importing some libraries and creating an experiment to track the runs in your workspace. A workspace can have multiple experiments, and all the users that have access to the workspace can collaborate on them. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612965838618
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"\n",
"from azureml.core import Workspace\n",
"from azureml.core import Experiment\n",
"\n",
"# connect to your workspace\n",
"ws = Workspace.from_config()\n",
"\n",
"experiment_name = \"get-started-with-jobsubmission-tutorial\"\n",
"exp = Experiment(workspace=ws, name=experiment_name)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### The MNIST dataset\n",
"\n",
"Use Azure Open Datasets to get the raw MNIST data files. [Azure Open Datasets](https://docs.microsoft.com/azure/open-datasets/overview-what-are-open-datasets) are curated public datasets that you can use to add scenario-specific features to machine learning solutions for more accurate models. Each dataset has a corresponding class, `MNIST` in this case, to retrieve the data in different ways.\n",
"\n",
"Follow this [how-to](https://aka.ms/azureml/howto/createdatasets) if you want to learn more about Datasets and how to use them.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612965850391
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"from azureml.opendatasets import MNIST\n",
"\n",
"mnist_file_dataset = MNIST.get_file_dataset()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Define the Environment\n",
"An Environment defines Python packages, environment variables, and Docker settings that are used in machine learning experiments. Here you will be using a curated environment that has already been made available through the workspace. \n",
"\n",
"Read [this article](https://docs.microsoft.com/azure/machine-learning/how-to-use-environments) if you want to learn more about Environments and how to use them."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612965877458
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"from azureml.core.environment import Environment\n",
"\n",
"# use a curated environment that has already been built for you\n",
"\n",
"env = Environment.get(workspace=ws, \n",
" name=\"AzureML-Scikit-learn0.24-Cuda11-OpenMpi4.1.0-py36\", \n",
" version=1)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Configure the training job\n",
"\n",
"Create a [ScriptRunConfig](https://docs.microsoft.com/python/api/azureml-core/azureml.core.script_run_config.scriptrunconfig?view=azure-ml-py) object to specify the configuration details of your training job, including your training script, environment to use, and the compute target to run on. Configure the ScriptRunConfig by specifying:\n",
"\n",
"* The directory that contains your scripts. All the files in this directory are uploaded into the cluster nodes for execution. \n",
"* The compute target. In this case you will point to local compute\n",
"* The training script name, train.py\n",
"* An environment that contains the libraries needed to run the script\n",
"* Arguments required from the training script. \n",
"\n",
"In this run we will be submitting to \"local\", which is the compute instance you are running this notebook. If you have another compute target (for example: AKS, Azure ML Compute Cluster, Azure Databricks, etc) then you just need to change the `compute_target` argument below. You can learn more about other compute targets [here](https://docs.microsoft.com/azure/machine-learning/how-to-set-up-training-targets). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612965882781
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"from azureml.core import ScriptRunConfig\n",
"\n",
"args = [\"--data-folder\", mnist_file_dataset.as_mount(), \"--regularization\", 0.5]\n",
"\n",
"src = ScriptRunConfig(\n",
" source_directory=\"src\",\n",
" script=\"train.py\",\n",
" arguments=args,\n",
" compute_target=\"local\",\n",
" environment=env,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"### Submit the job\n",
"\n",
"Run the experiment by submitting the ScriptRunConfig object. After this there are many options for monitoring your run. Once submitted, you can either navigate to the experiment \"get-started-with-jobsubmission-tutorial\" in the left menu item __Experiments__ to monitor the run, or you can monitor the run inline as the `run.wait_for_completion(show_output=True)` will stream the logs of the run. You will see that the environment is built for you to ensure reproducibility - this adds a couple of minutes to the run time. On subsequent runs, the environment is re-used making the runtime shorter."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612965911435
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"run = exp.submit(config=src)\n",
"run.wait_for_completion(show_output=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Register model\n",
"\n",
"The training script used the MLflow autologging feature and therefore the model was captured and stored on your behalf. Below we register the model into the Azure Machine Learning Model registry, which lets you keep track of all the models in your Azure Machine Learning workspace.\n",
"\n",
"Models are identified by name and version. Each time you register a model with the same name as an existing one, the registry assumes that it's a new version. The version is incremented, and the new model is registered under the same name.\n",
"\n",
"When you register the model, you can provide additional metadata tags and then use the tags when you search for models."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1612966068862
},
"jupyter": {
"outputs_hidden": false,
"source_hidden": false
},
"nteract": {
"transient": {
"deleting": false
}
}
},
"outputs": [],
"source": [
"# register model\n",
"model = run.register_model(\n",
" model_name=\"sklearn_mnist\", model_path=\"model/model.pkl\"\n",
")\n",
"print(model.name, model.id, model.version, sep=\"\\t\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You will now be able to see the model in the regsitry by selecting __Models__ in the left-hand menu of the Azure Machine Learning Studio."
]
},
{
"cell_type": "markdown",
"metadata": {
"nteract": {
"transient": {
"deleting": false
}
}
},
"source": [
"## Control Cost\n",
"\n",
"If you want to control cost you can stop the compute instance this notebook is running on by clicking the \"Stop compute\" button next to the status dropdown in the menu above.\n",
"\n",
" ## Next Steps\n",
"\n",
"In this quickstart, you have seen how to run jobs-based machine learning code in Azure Machine Learning. \n",
"\n",
"It is also possible to use automated machine learning in Azure Machine Learning service to find the best model in an automated fashion. To see how this works, we recommend that you follow the next quickstart in this series, [**Fraud Classification using Automated ML**](../quickstart-azureml-automl/quickstart-azureml-automl.ipynb). This quickstart is focused on AutoML using the Python SDK."
]
}
],
"metadata": {
"authors": [
{
"name": "cewidste"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
},
"notice": "Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License.",
"nteract": {
"version": "nteract-front-end@1.0.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

12
tutorials/compute-instance-quickstarts/quickstart-azureml-python-sdk/quickstart-azureml-python-sdk.yml Normal file
View File

@@ -0,0 +1,12 @@
name: quickstart-azureml-python-sdk
dependencies:
- pip:
- azureml-sdk
- sklearn
- numpy
- matplotlib
- joblib
- uuid
- requests
- azureml-opendatasets
- azureml-widgets

72
tutorials/compute-instance-quickstarts/quickstart-azureml-python-sdk/src/train.py Normal file
View File

@@ -0,0 +1,72 @@
import argparse
import os
import numpy as np
import glob
# import joblib
import mlflow
from sklearn.linear_model import LogisticRegression
from utils import load_data
# let user feed in 2 parameters, the dataset to mount or download,
# and the regularization rate of the logistic regression model
parser = argparse.ArgumentParser()
parser.add_argument(
"--data-folder", type=str, dest="data_folder", help="data folder mounting point"
)
parser.add_argument(
"--regularization", type=float, dest="reg", default=0.01, help="regularization rate"
)
args = parser.parse_args()
data_folder = args.data_folder
print("Data folder:", data_folder)
# load train and test set into numpy arrays
# note we scale the pixel intensity values to 0-1 (by dividing it with 255.0) so the model can converge faster.
X_train = (
load_data(
glob.glob(
os.path.join(data_folder, "**/train-images-idx3-ubyte.gz"), recursive=True
)[0],
False,
) / 255.0
)
X_test = (
load_data(
glob.glob(
os.path.join(data_folder, "**/t10k-images-idx3-ubyte.gz"), recursive=True
)[0],
False,
) / 255.0
)
y_train = load_data(
glob.glob(
os.path.join(data_folder, "**/train-labels-idx1-ubyte.gz"), recursive=True
)[0],
True,
).reshape(-1)
y_test = load_data(
glob.glob(
os.path.join(data_folder, "**/t10k-labels-idx1-ubyte.gz"), recursive=True
)[0],
True,
).reshape(-1)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep="\n")
# use mlflow autologging
mlflow.autolog()
print("Train a logistic regression model with regularization rate of", args.reg)
clf = LogisticRegression(
C=1.0 / args.reg, solver="liblinear", multi_class="auto", random_state=42
)
clf.fit(X_train, y_train)
print("Predict the test set")
y_hat = clf.predict(X_test)
# calculate accuracy on the prediction
acc = np.average(y_hat == y_test)
print("Accuracy is", acc)

24
tutorials/compute-instance-quickstarts/quickstart-azureml-python-sdk/src/utils.py Normal file
View File

@@ -0,0 +1,24 @@
import gzip
import numpy as np
import struct
# load compressed MNIST gz files and return numpy arrays
def load_data(filename, label=False):
with gzip.open(filename) as gz:
struct.unpack("I", gz.read(4))
n_items = struct.unpack(">I", gz.read(4))
if not label:
n_rows = struct.unpack(">I", gz.read(4))[0]
n_cols = struct.unpack(">I", gz.read(4))[0]
res = np.frombuffer(gz.read(n_items[0] * n_rows * n_cols), dtype=np.uint8)
res = res.reshape(n_items[0], n_rows * n_cols)
else:
res = np.frombuffer(gz.read(n_items[0]), dtype=np.uint8)
res = res.reshape(n_items[0], 1)
return res
# one-hot encode a 1-D array
def one_hot_encode(array, num_of_classes):
return np.eye(num_of_classes)[array.reshape(-1)]

BIN
tutorials/create-first-ml-experiment/imgs/experiment_main.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 68 KiB

BIN
tutorials/create-first-ml-experiment/imgs/model_download.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 26 KiB

395
tutorials/create-first-ml-experiment/tutorial-1st-experiment-sdk-train.ipynb Normal file
View File

@@ -0,0 +1,395 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/tutorial-quickstart-train-model.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial: Train your first model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This tutorial is **part two of a two-part tutorial series**. In the previous tutorial, you created a workspace and chose a development environment. In this tutorial, you learn the foundational design patterns in Azure Machine Learning service, and train a simple scikit-learn model based on the diabetes data set. After completing this tutorial, you will have the practical knowledge of the SDK to scale up to developing more-complex experiments and workflows. \n",
"\n",
"In this tutorial, you learn the following tasks:\n",
"\n",
"> * Connect your workspace and create an experiment \n",
"> * Load data and train a scikit-learn model\n",
"> * View training results in the studio\n",
"> * Retrieve the best model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"The only prerequisite is to run the previous tutorial, Setup environment and workspace."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Connect workspace and create experiment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Import the `Workspace` class, and load your subscription information from the file `config.json` using the function `from_config().` This looks for the JSON file in the current directory by default, but you can also specify a path parameter to point to the file using `from_config(path=\"your/file/path\")`. If you are running this notebook in a cloud notebook server in your workspace, the file is automatically in the root directory.\n",
"\n",
"If the following code asks for additional authentication, simply paste the link in a browser and enter the authentication token. In addition, if you have more than one tenant linked to your user, you will need to add the following lines:\n",
"```\n",
"from azureml.core.authentication import InteractiveLoginAuthentication\n",
"interactive_auth = InteractiveLoginAuthentication(tenant_id=\"your-tenant-id\")\n",
"Additional details on authentication can be found here: https://aka.ms/aml-notebook-auth \n",
"```\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core import Workspace\n",
"ws = Workspace.from_config()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now create an experiment in your workspace. An experiment is another foundational cloud resource that represents a collection of trials (individual model runs). In this tutorial you use the experiment to create runs and track your model training in the Azure Machine Learning studio. Parameters include your workspace reference, and a string name for the experiment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core import Experiment\n",
"experiment = Experiment(workspace=ws, name=\"diabetes-experiment\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load data and prepare for training"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For this tutorial, you use the diabetes data set, which uses features like age, gender, and BMI to predict diabetes disease progression. Load the data from the Azure Open Datasets class, and split it into training and test sets using `train_test_split()`. This function segregates the data so the model has unseen data to use for testing following training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.opendatasets import Diabetes\n",
"from sklearn.model_selection import train_test_split\n",
"\n",
"x_df = Diabetes.get_tabular_dataset().to_pandas_dataframe().dropna()\n",
"y_df = x_df.pop(\"Y\")\n",
"\n",
"X_train, X_test, y_train, y_test = train_test_split(x_df, y_df, test_size=0.2, random_state=66)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train a model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Training a simple scikit-learn model can easily be done locally for small-scale training, but when training many iterations with dozens of different feature permutations and hyperparameter settings, it is easy to lose track of what models you've trained and how you trained them. The following design pattern shows how to leverage the SDK to easily keep track of your training in the cloud.\n",
"\n",
"Build a script that trains ridge models in a loop through different hyperparameter alpha values."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.linear_model import Ridge\n",
"from sklearn.metrics import mean_squared_error\n",
"import joblib\n",
"import math\n",
"\n",
"alphas = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]\n",
"\n",
"for alpha in alphas:\n",
" run = experiment.start_logging()\n",
" run.log(\"alpha_value\", alpha)\n",
" \n",
" model = Ridge(alpha=alpha)\n",
" model.fit(X=X_train, y=y_train)\n",
" y_pred = model.predict(X=X_test)\n",
" rmse = math.sqrt(mean_squared_error(y_true=y_test, y_pred=y_pred))\n",
" run.log(\"rmse\", rmse)\n",
" \n",
" model_name = \"model_alpha_\" + str(alpha) + \".pkl\"\n",
" filename = \"outputs/\" + model_name\n",
" \n",
" joblib.dump(value=model, filename=filename)\n",
" run.upload_file(name=model_name, path_or_stream=filename)\n",
" run.complete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The above code accomplishes the following:\n",
"\n",
"1. For each alpha hyperparameter value in the `alphas` array, a new run is created within the experiment. The alpha value is logged to differentiate between each run.\n",
"1. In each run, a Ridge model is instantiated, trained, and used to run predictions. The root-mean-squared-error is calculated for the actual versus predicted values, and then logged to the run. At this point the run has metadata attached for both the alpha value and the rmse accuracy.\n",
"1. Next, the model for each run is serialized and uploaded to the run. This allows you to download the model file from the run in the studio.\n",
"1. At the end of each iteration the run is completed by calling `run.complete()`.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After the training has completed, call the `experiment` variable to fetch a link to the experiment in the studio."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"experiment"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## View training results in studio"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Following the **Link to Azure Machine Learning studio** takes you to the main experiment page. Here you see all the individual runs in the experiment. Any custom-logged values (`alpha_value` and `rmse`, in this case) become fields for each run, and also become available for the charts and tiles at the top of the experiment page. To add a logged metric to a chart or tile, hover over it, click the edit button, and find your custom-logged metric.\n",
"\n",
"When training models at scale over hundreds and thousands of runs, this page makes it easy to see every model you trained, specifically how they were trained, and how your unique metrics have changed over time."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Main Experiment page in the studio](./imgs/experiment_main.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Select a run number link in the `RUN NUMBER` column to see the page for an individual run. The default tab **Details** shows you more-detailed information on each run. Navigate to the **Outputs + logs** tab, and you see the `.pkl` file for the model that was uploaded to the run during each training iteration. Here you can download the model file, rather than having to retrain it manually."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Run details page in the studio](./imgs/model_download.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Get the best model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In addition to being able to download model files from the experiment in the studio, you can also download them programmatically. The following code iterates through each run in the experiment, and accesses both the logged run metrics and the run details (which contains the run_id). This keeps track of the best run, in this case the run with the lowest root-mean-squared-error."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"minimum_rmse_runid = None\n",
"minimum_rmse = None\n",
"\n",
"for run in experiment.get_runs():\n",
" run_metrics = run.get_metrics()\n",
" run_details = run.get_details()\n",
" # each logged metric becomes a key in this returned dict\n",
" run_rmse = run_metrics[\"rmse\"]\n",
" run_id = run_details[\"runId\"]\n",
" \n",
" if minimum_rmse is None:\n",
" minimum_rmse = run_rmse\n",
" minimum_rmse_runid = run_id\n",
" else:\n",
" if run_rmse < minimum_rmse:\n",
" minimum_rmse = run_rmse\n",
" minimum_rmse_runid = run_id\n",
"\n",
"print(\"Best run_id: \" + minimum_rmse_runid)\n",
"print(\"Best run_id rmse: \" + str(minimum_rmse)) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the best run id to fetch the individual run using the `Run` constructor along with the experiment object. Then call `get_file_names()` to see all the files available for download from this run. In this case, you only uploaded one file for each run during training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core import Run\n",
"best_run = Run(experiment=experiment, run_id=minimum_rmse_runid)\n",
"print(best_run.get_file_names())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Call `download()` on the run object, specifying the model file name to download. By default this function downloads to the current directory."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"best_run.download_file(name=\"model_alpha_0.1.pkl\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up resources\n",
"\n",
"Do not complete this section if you plan on running other Azure Machine Learning service tutorials.\n",
"\n",
"### Stop the notebook VM\n",
"\n",
"If you used a cloud notebook server, stop the VM when you are not using it to reduce cost.\n",
"\n",
"1. In your workspace, select **Compute**.\n",
"\n",
"1. Select the **Notebook VMs** tab in the compute page.\n",
"\n",
"1. From the list, select the VM.\n",
"\n",
"1. Select **Stop**.\n",
"\n",
"1. When you're ready to use the server again, select **Start**.\n",
"\n",
"### Delete everything\n",
"\n",
"If you don't plan to use the resources you created, delete them, so you don't incur any charges:\n",
"\n",
"1. In the Azure portal, select **Resource groups** on the far left.\n",
"\n",
"1. From the list, select the resource group you created.\n",
"\n",
"1. Select **Delete resource group**.\n",
"\n",
"1. Enter the resource group name. Then select **Delete**.\n",
"\n",
"You can also keep the resource group but delete a single workspace. Display the workspace properties and select **Delete**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"In this tutorial, you did the following tasks:\n",
"\n",
"> * Connected your workspace and created an experiment\n",
"> * Loaded data and trained scikit-learn models\n",
"> * Viewed training results in the studio and retrieved models\n",
"\n",
"[Deploy your model](https://docs.microsoft.com/azure/machine-learning/service/tutorial-deploy-models-with-aml) with Azure Machine Learning.\n",
"Learn how to develop [automated machine learning](https://docs.microsoft.com/azure/machine-learning/service/tutorial-auto-train-models) experiments."
]
}
],
"metadata": {
"authors": [
{
"name": "trbye"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
},
"msauthor": "trbye",
"network_required": false
},
"nbformat": 4,
"nbformat_minor": 2
}

6
tutorials/create-first-ml-experiment/tutorial-1st-experiment-sdk-train.yml Normal file
View File

@@ -0,0 +1,6 @@
name: tutorial-1st-experiment-sdk-train
dependencies:
- pip:
- azureml-sdk
- sklearn
- azureml-opendatasets

695
tutorials/image-classification-mnist-data/img-classification-part1-training.ipynb Normal file
View File

@@ -0,0 +1,695 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
"\n",
"Licensed under the MIT License."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial #1: Train an image classification model with Azure Machine Learning\n",
"\n",
"In this tutorial, you train a machine learning model on remote compute resources. You'll use the training and deployment workflow for Azure Machine Learning service (preview) in a Python Jupyter notebook. You can then use the notebook as a template to train your own machine learning model with your own data. This tutorial is **part one of a two-part tutorial series**. \n",
"\n",
"This tutorial trains a simple logistic regression using the [MNIST](https://docs.microsoft.com/azure/open-datasets/dataset-mnist) dataset and [scikit-learn](http://scikit-learn.org) with Azure Machine Learning. MNIST is a popular dataset consisting of 70,000 grayscale images. Each image is a handwritten digit of 28x28 pixels, representing a number from 0 to 9. The goal is to create a multi-class classifier to identify the digit a given image represents. \n",
"\n",
"Learn how to:\n",
"\n",
"> * Set up your development environment\n",
"> * Access and examine the data\n",
"> * Train a simple logistic regression model on a remote cluster\n",
"> * Review training results, find and register the best model\n",
"\n",
"You'll learn how to select a model and deploy it in [part two of this tutorial](deploy-models.ipynb) later. \n",
"\n",
"## Prerequisites\n",
"\n",
"See prerequisites in the [Azure Machine Learning documentation](https://docs.microsoft.com/azure/machine-learning/service/tutorial-train-models-with-aml#prerequisites).\n",
"\n",
"On the computer running this notebook, conda install matplotlib, numpy, scikit-learn=0.22.1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set up your development environment\n",
"\n",
"All the setup for your development work can be accomplished in a Python notebook. Setup includes:\n",
"\n",
"* Importing Python packages\n",
"* Connecting to a workspace to enable communication between your local computer and remote resources\n",
"* Creating an experiment to track all your runs\n",
"* Creating a remote compute target to use for training\n",
"\n",
"### Import packages\n",
"\n",
"Import Python packages you need in this session. Also display the Azure Machine Learning SDK version."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"check version"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"import azureml.core\n",
"from azureml.core import Workspace\n",
"\n",
"# check core SDK version number\n",
"print(\"Azure ML SDK Version: \", azureml.core.VERSION)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Connect to workspace\n",
"\n",
"Create a workspace object from the existing workspace. `Workspace.from_config()` reads the file **config.json** and loads the details into an object named `ws`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"load workspace"
]
},
"outputs": [],
"source": [
"# load workspace configuration from the config.json file in the current folder.\n",
"ws = Workspace.from_config()\n",
"print(ws.name, ws.location, ws.resource_group, sep='\\t')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create experiment\n",
"\n",
"Create an experiment to track the runs in your workspace. A workspace can have muliple experiments. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"create experiment"
]
},
"outputs": [],
"source": [
"experiment_name = 'Tutorial-sklearn-mnist'\n",
"\n",
"from azureml.core import Experiment\n",
"exp = Experiment(workspace=ws, name=experiment_name)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create or Attach existing compute resource\n",
"\n",
"> Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist.\n",
"\n",
"By using Azure Machine Learning Compute, a managed service, data scientists can train machine learning models on clusters of Azure virtual machines. Examples include VMs with GPU support. In this tutorial, you create Azure Machine Learning Compute as your training environment. You will submit Python code to run on this VM later in the tutorial. \n",
"The code below creates the compute clusters for you if they don't already exist in your workspace.\n",
"\n",
"**Creation of compute takes approximately 5 minutes.** If the AmlCompute with that name is already in your workspace the code will skip the creation process."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"create mlc",
"amlcompute"
]
},
"outputs": [],
"source": [
"from azureml.core.compute import AmlCompute\n",
"from azureml.core.compute import ComputeTarget\n",
"import os\n",
"\n",
"# choose a name for your cluster\n",
"compute_name = os.environ.get(\"AML_COMPUTE_CLUSTER_NAME\", \"cpu-cluster\")\n",
"compute_min_nodes = os.environ.get(\"AML_COMPUTE_CLUSTER_MIN_NODES\", 0)\n",
"compute_max_nodes = os.environ.get(\"AML_COMPUTE_CLUSTER_MAX_NODES\", 4)\n",
"\n",
"# This example uses CPU VM. For using GPU VM, set SKU to STANDARD_NC6\n",
"vm_size = os.environ.get(\"AML_COMPUTE_CLUSTER_SKU\", \"STANDARD_D2_V2\")\n",
"\n",
"\n",
"if compute_name in ws.compute_targets:\n",
" compute_target = ws.compute_targets[compute_name]\n",
" if compute_target and type(compute_target) is AmlCompute:\n",
" print(\"found compute target: \" + compute_name)\n",
"else:\n",
" print(\"creating new compute target...\")\n",
" provisioning_config = AmlCompute.provisioning_configuration(vm_size = vm_size,\n",
" min_nodes = compute_min_nodes, \n",
" max_nodes = compute_max_nodes)\n",
"\n",
" # create the cluster\n",
" compute_target = ComputeTarget.create(ws, compute_name, provisioning_config)\n",
" \n",
" # can poll for a minimum number of nodes and for a specific timeout. \n",
" # if no min node count is provided it will use the scale settings for the cluster\n",
" compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)\n",
" \n",
" # For a more detailed view of current AmlCompute status, use get_status()\n",
" print(compute_target.get_status().serialize())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You now have the necessary packages and compute resources to train a model in the cloud. \n",
"\n",
"## Explore data\n",
"\n",
"Before you train a model, you need to understand the data that you are using to train it. In this section you learn how to:\n",
"\n",
"* Download the MNIST dataset\n",
"* Display some sample images\n",
"\n",
"### Download the MNIST dataset\n",
"\n",
"Use Azure Open Datasets to get the raw MNIST data files. [Azure Open Datasets](https://docs.microsoft.com/azure/open-datasets/overview-what-are-open-datasets) are curated public datasets that you can use to add scenario-specific features to machine learning solutions for more accurate models. Each dataset has a corrseponding class, `MNIST` in this case, to retrieve the data in different ways.\n",
"\n",
"This code retrieves the data as a `FileDataset` object, which is a subclass of `Dataset`. A `FileDataset` references single or multiple files of any format in your datastores or public urls. The class provides you with the ability to download or mount the files to your compute by creating a reference to the data source location. Additionally, you register the Dataset to your workspace for easy retrieval during training.\n",
"\n",
"Follow the [how-to](https://aka.ms/azureml/howto/createdatasets) to learn more about Datasets and their usage in the SDK."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core import Dataset\n",
"from azureml.opendatasets import MNIST\n",
"\n",
"data_folder = os.path.join(os.getcwd(), 'data')\n",
"os.makedirs(data_folder, exist_ok=True)\n",
"\n",
"mnist_file_dataset = MNIST.get_file_dataset()\n",
"mnist_file_dataset.download(data_folder, overwrite=True)\n",
"\n",
"mnist_file_dataset = mnist_file_dataset.register(workspace=ws,\n",
" name='mnist_opendataset',\n",
" description='training and test dataset',\n",
" create_new_version=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Display some sample images\n",
"\n",
"Load the compressed files into `numpy` arrays. Then use `matplotlib` to plot 30 random images from the dataset with their labels above them. Note this step requires a `load_data` function that's included in an `utils.py` file. This file is included in the sample folder. Please make sure it is placed in the same folder as this notebook. The `load_data` function simply parses the compresse files into numpy arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# make sure utils.py is in the same directory as this code\n",
"from utils import load_data\n",
"import glob\n",
"\n",
"\n",
"# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the model converge faster.\n",
"X_train = load_data(glob.glob(os.path.join(data_folder,\"**/train-images-idx3-ubyte.gz\"), recursive=True)[0], False) / 255.0\n",
"X_test = load_data(glob.glob(os.path.join(data_folder,\"**/t10k-images-idx3-ubyte.gz\"), recursive=True)[0], False) / 255.0\n",
"y_train = load_data(glob.glob(os.path.join(data_folder,\"**/train-labels-idx1-ubyte.gz\"), recursive=True)[0], True).reshape(-1)\n",
"y_test = load_data(glob.glob(os.path.join(data_folder,\"**/t10k-labels-idx1-ubyte.gz\"), recursive=True)[0], True).reshape(-1)\n",
"\n",
"\n",
"# now let's show some randomly chosen images from the traininng set.\n",
"count = 0\n",
"sample_size = 30\n",
"plt.figure(figsize = (16, 6))\n",
"for i in np.random.permutation(X_train.shape[0])[:sample_size]:\n",
" count = count + 1\n",
" plt.subplot(1, sample_size, count)\n",
" plt.axhline('')\n",
" plt.axvline('')\n",
" plt.text(x=10, y=-10, s=y_train[i], fontsize=18)\n",
" plt.imshow(X_train[i].reshape(28, 28), cmap=plt.cm.Greys)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train on a remote cluster\n",
"\n",
"For this task, you submit the job to run on the remote training cluster you set up earlier. To submit a job you:\n",
"* Create a directory\n",
"* Create a training script\n",
"* Create a script run configuration\n",
"* Submit the job \n",
"\n",
"### Create a directory\n",
"\n",
"Create a directory to deliver the necessary code from your computer to the remote resource."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"script_folder = os.path.join(os.getcwd(), \"sklearn-mnist\")\n",
"os.makedirs(script_folder, exist_ok=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create a training script\n",
"\n",
"To submit the job to the cluster, first create a training script. Run the following code to create the training script called `train.py` in the directory you just created. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile $script_folder/train.py\n",
"\n",
"import argparse\n",
"import os\n",
"import numpy as np\n",
"import glob\n",
"\n",
"from sklearn.linear_model import LogisticRegression\n",
"import joblib\n",
"\n",
"from azureml.core import Run\n",
"from utils import load_data\n",
"\n",
"# let user feed in 2 parameters, the dataset to mount or download, and the regularization rate of the logistic regression model\n",
"parser = argparse.ArgumentParser()\n",
"parser.add_argument('--data-folder', type=str, dest='data_folder', help='data folder mounting point')\n",
"parser.add_argument('--regularization', type=float, dest='reg', default=0.01, help='regularization rate')\n",
"args = parser.parse_args()\n",
"\n",
"data_folder = args.data_folder\n",
"print('Data folder:', data_folder)\n",
"\n",
"# load train and test set into numpy arrays\n",
"# note we scale the pixel intensity values to 0-1 (by dividing it with 255.0) so the model can converge faster.\n",
"X_train = load_data(glob.glob(os.path.join(data_folder, '**/train-images-idx3-ubyte.gz'), recursive=True)[0], False) / 255.0\n",
"X_test = load_data(glob.glob(os.path.join(data_folder, '**/t10k-images-idx3-ubyte.gz'), recursive=True)[0], False) / 255.0\n",
"y_train = load_data(glob.glob(os.path.join(data_folder, '**/train-labels-idx1-ubyte.gz'), recursive=True)[0], True).reshape(-1)\n",
"y_test = load_data(glob.glob(os.path.join(data_folder, '**/t10k-labels-idx1-ubyte.gz'), recursive=True)[0], True).reshape(-1)\n",
"\n",
"print(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep = '\\n')\n",
"\n",
"# get hold of the current run\n",
"run = Run.get_context()\n",
"\n",
"print('Train a logistic regression model with regularization rate of', args.reg)\n",
"clf = LogisticRegression(C=1.0/args.reg, solver=\"liblinear\", multi_class=\"auto\", random_state=42)\n",
"clf.fit(X_train, y_train)\n",
"\n",
"print('Predict the test set')\n",
"y_hat = clf.predict(X_test)\n",
"\n",
"# calculate accuracy on the prediction\n",
"acc = np.average(y_hat == y_test)\n",
"print('Accuracy is', acc)\n",
"\n",
"run.log('regularization rate', np.float(args.reg))\n",
"run.log('accuracy', np.float(acc))\n",
"\n",
"os.makedirs('outputs', exist_ok=True)\n",
"# note file saved in the outputs folder is automatically uploaded into experiment record\n",
"joblib.dump(value=clf, filename='outputs/sklearn_mnist_model.pkl')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notice how the script gets data and saves models:\n",
"\n",
"+ The training script reads an argument to find the directory containing the data. When you submit the job later, you point to the dataset for this argument:\n",
"`parser.add_argument('--data-folder', type=str, dest='data_folder', help='data directory mounting point')`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"+ The training script saves your model into a directory named outputs. <br/>\n",
"`joblib.dump(value=clf, filename='outputs/sklearn_mnist_model.pkl')`<br/>\n",
"Anything written in this directory is automatically uploaded into your workspace. You'll access your model from this directory later in the tutorial."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The file `utils.py` is referenced from the training script to load the dataset correctly. Copy this script into the script folder so that it can be accessed along with the training script on the remote resource."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import shutil\n",
"shutil.copy('utils.py', script_folder)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configure the training job\n",
"\n",
"Create a ScriptRunConfig object to specify the configuration details of your training job, including your training script, environment to use, and the compute target to run on. Configure the ScriptRunConfig by specifying:\n",
"\n",
"* The directory that contains your scripts. All the files in this directory are uploaded into the cluster nodes for execution. \n",
"* The compute target. In this case you will use the AmlCompute you created\n",
"* The training script name, train.py\n",
"* An environment that contains the libraries needed to run the script\n",
"* Arguments required from the training script. \n",
"\n",
"In this tutorial, the target is AmlCompute. All files in the script folder are uploaded into the cluster nodes for execution. The data_folder is set to use the dataset.\n",
"\n",
"First, create the environment that contains: the scikit-learn library, azureml-dataset-runtime required for accessing the dataset, and azureml-defaults which contains the dependencies for logging metrics. The azureml-defaults also contains the dependencies required for deploying the model as a web service later in the part 2 of the tutorial.\n",
"\n",
"Once the environment is defined, register it with the Workspace to re-use it in part 2 of the tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.environment import Environment\n",
"from azureml.core.conda_dependencies import CondaDependencies\n",
"\n",
"# to install required packages\n",
"env = Environment('tutorial-env')\n",
"cd = CondaDependencies.create(pip_packages=['azureml-dataset-runtime[pandas,fuse]', 'azureml-defaults'], conda_packages = ['scikit-learn==0.22.1'])\n",
"\n",
"env.python.conda_dependencies = cd\n",
"\n",
"# Register environment to re-use later\n",
"env.register(workspace = ws)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then, create the ScriptRunConfig by specifying the training script, compute target and environment."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"configure estimator"
]
},
"outputs": [],
"source": [
"from azureml.core import ScriptRunConfig\n",
"\n",
"args = ['--data-folder', mnist_file_dataset.as_mount(), '--regularization', 0.5]\n",
"\n",
"src = ScriptRunConfig(source_directory=script_folder,\n",
" script='train.py', \n",
" arguments=args,\n",
" compute_target=compute_target,\n",
" environment=env)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Submit the job to the cluster\n",
"\n",
"Run the experiment by submitting the ScriptRunConfig object. And you can navigate to Azure portal to monitor the run."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"remote run",
"amlcompute",
"scikit-learn"
]
},
"outputs": [],
"source": [
"run = exp.submit(config=src)\n",
"run"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since the call is asynchronous, it returns a **Preparing** or **Running** state as soon as the job is started.\n",
"\n",
"## Monitor a remote run\n",
"\n",
"In total, the first run takes **approximately 10 minutes**. But for subsequent runs, as long as the dependencies in the Azure ML environment don't change, the same image is reused and hence the container start up time is much faster.\n",
"\n",
"Here is what's happening while you wait:\n",
"\n",
"- **Image creation**: A Docker image is created matching the Python environment specified by the Azure ML environment. The image is built and stored in the ACR (Azure Container Registry) associated with your workspace. Image creation and uploading takes **about 5 minutes**. \n",
"\n",
" This stage happens once for each Python environment since the container is cached for subsequent runs. During image creation, logs are streamed to the run history. You can monitor the image creation progress using these logs.\n",
"\n",
"- **Scaling**: If the remote cluster requires more nodes to execute the run than currently available, additional nodes are added automatically. Scaling typically takes **about 5 minutes.**\n",
"\n",
"- **Running**: In this stage, the necessary scripts and files are sent to the compute target, then data stores are mounted/copied, then the entry_script is run. While the job is running, stdout and the files in the ./logs directory are streamed to the run history. You can monitor the run's progress using these logs.\n",
"\n",
"- **Post-Processing**: The ./outputs directory of the run is copied over to the run history in your workspace so you can access these results.\n",
"\n",
"\n",
"You can check the progress of a running job in multiple ways. This tutorial uses a Jupyter widget as well as a `wait_for_completion` method. \n",
"\n",
"### Jupyter widget\n",
"\n",
"Watch the progress of the run with a Jupyter widget. Like the run submission, the widget is asynchronous and provides live updates every 10-15 seconds until the job completes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"use notebook widget"
]
},
"outputs": [],
"source": [
"from azureml.widgets import RunDetails\n",
"RunDetails(run).show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By the way, if you need to cancel a run, you can follow [these instructions](https://aka.ms/aml-docs-cancel-run)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get log results upon completion\n",
"\n",
"Model training happens in the background. You can use `wait_for_completion` to block and wait until the model has completed training before running more code. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"remote run",
"amlcompute",
"scikit-learn"
]
},
"outputs": [],
"source": [
"# specify show_output to True for a verbose log\n",
"run.wait_for_completion(show_output=True) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Display run results\n",
"\n",
"You now have a model trained on a remote cluster. Retrieve all the metrics logged during the run, including the accuracy of the model:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"get metrics"
]
},
"outputs": [],
"source": [
"print(run.get_metrics())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the next tutorial you will explore this model in more detail.\n",
"\n",
"## Register model\n",
"\n",
"The last step in the training script wrote the file `outputs/sklearn_mnist_model.pkl` in a directory named `outputs` in the VM of the cluster where the job is executed. `outputs` is a special directory in that all content in this directory is automatically uploaded to your workspace. This content appears in the run record in the experiment under your workspace. Hence, the model file is now also available in your workspace.\n",
"\n",
"You can see files associated with that run."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"query history"
]
},
"outputs": [],
"source": [
"print(run.get_file_names())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Register the model in the workspace so that you (or other collaborators) can later query, examine, and deploy this model."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"register model from history"
]
},
"outputs": [],
"source": [
"# register model \n",
"model = run.register_model(model_name='sklearn_mnist', model_path='outputs/sklearn_mnist_model.pkl')\n",
"print(model.name, model.id, model.version, sep='\\t')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"In this Azure Machine Learning tutorial, you used Python to:\n",
"\n",
"> * Set up your development environment\n",
"> * Access and examine the data\n",
"> * Train multiple models on a remote cluster using the popular scikit-learn machine learning library\n",
"> * Review training details and register the best model\n",
"\n",
"You are ready to deploy this registered model using the instructions in the next part of the tutorial series:\n",
"\n",
"> [Tutorial 2 - Deploy models](img-classification-part2-deploy.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/img-classification-part1-training.png)"
]
}
],
"metadata": {
"authors": [
{
"name": "maxluk"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
},
"msauthor": "roastala",
"network_required": false
},
"nbformat": 4,
"nbformat_minor": 2
}

10
tutorials/image-classification-mnist-data/img-classification-part1-training.yml Normal file
View File

@@ -0,0 +1,10 @@
name: img-classification-part1-training
dependencies:
- pip:
- azureml-sdk
- azureml-widgets
- matplotlib
- sklearn
- pandas
- azureml-opendatasets
- azureml-widgets

575
tutorials/image-classification-mnist-data/img-classification-part2-deploy.ipynb Normal file
View File

@@ -0,0 +1,575 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
"\n",
"Licensed under the MIT License."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial #2: Deploy an image classification model in Azure Container Instance (ACI)\n",
"\n",
"This tutorial is **part two of a two-part tutorial series**. In the [previous tutorial](img-classification-part1-training.ipynb), you trained machine learning models and then registered a model in your workspace on the cloud. \n",
"\n",
"Now, you're ready to deploy the model as a web service in [Azure Container Instances](https://docs.microsoft.com/azure/container-instances/) (ACI). A web service is an image, in this case a Docker image, that encapsulates the scoring logic and the model itself. \n",
"\n",
"In this part of the tutorial, you use Azure Machine Learning service (Preview) to:\n",
"\n",
"> * Set up your testing environment\n",
"> * Retrieve the model from your workspace\n",
"> * Test the model locally\n",
"> * Deploy the model to ACI\n",
"> * Test the deployed model\n",
"\n",
"ACI is a great solution for testing and understanding the workflow. For scalable production deployments, consider using Azure Kubernetes Service. For more information, see [how to deploy and where](https://docs.microsoft.com/azure/machine-learning/service/how-to-deploy-and-where).\n",
"\n",
"\n",
"## Prerequisites\n",
"\n",
"Complete the model training in the [Tutorial #1: Train an image classification model with Azure Machine Learning](train-models.ipynb) notebook. \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# If you did NOT complete the tutorial, you can instead run this cell \n",
"# This will register a model and download the data needed for this tutorial\n",
"# These prerequisites are created in the training tutorial\n",
"# Feel free to skip this cell if you completed the training tutorial \n",
"\n",
"# register a model\n",
"from azureml.core import Workspace\n",
"ws = Workspace.from_config()\n",
"\n",
"from azureml.core.model import Model\n",
"\n",
"model_name = \"sklearn_mnist\"\n",
"model = Model.register(model_path=\"sklearn_mnist_model.pkl\",\n",
" model_name=model_name,\n",
" tags={\"data\": \"mnist\", \"model\": \"classification\"},\n",
" description=\"Mnist handwriting recognition\",\n",
" workspace=ws)\n",
"\n",
"from azureml.core.environment import Environment\n",
"from azureml.core.conda_dependencies import CondaDependencies\n",
"\n",
"# to install required packages\n",
"env = Environment('tutorial-env')\n",
"cd = CondaDependencies.create(pip_packages=['azureml-dataset-runtime[pandas,fuse]', 'azureml-defaults'], conda_packages = ['scikit-learn==0.22.1'])\n",
"\n",
"env.python.conda_dependencies = cd\n",
"\n",
"# Register environment to re-use later\n",
"env.register(workspace = ws)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set up the environment\n",
"\n",
"Start by setting up a testing environment.\n",
"\n",
"### Import packages\n",
"\n",
"Import the Python packages needed for this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"check version"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
" \n",
"import azureml.core\n",
"\n",
"# display the core SDK version number\n",
"print(\"Azure ML SDK Version: \", azureml.core.VERSION)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy as web service\n",
"\n",
"Deploy the model as a web service hosted in ACI. \n",
"\n",
"To build the correct environment for ACI, provide the following:\n",
"* A scoring script to show how to use the model\n",
"* A configuration file to build the ACI\n",
"* The model you trained before\n",
"\n",
"### Create scoring script\n",
"\n",
"Create the scoring script, called score.py, used by the web service call to show how to use the model.\n",
"\n",
"You must include two required functions into the scoring script:\n",
"* The `init()` function, which typically loads the model into a global object. This function is run only once when the Docker container is started. \n",
"\n",
"* The `run(input_data)` function uses the model to predict a value based on the input data. Inputs and outputs to the run typically use JSON for serialization and de-serialization, but other formats are supported.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile score.py\n",
"import json\n",
"import numpy as np\n",
"import os\n",
"import pickle\n",
"import joblib\n",
"\n",
"def init():\n",
" global model\n",
" # AZUREML_MODEL_DIR is an environment variable created during deployment.\n",
" # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)\n",
" # For multiple models, it points to the folder containing all deployed models (./azureml-models)\n",
" model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_mnist_model.pkl')\n",
" model = joblib.load(model_path)\n",
"\n",
"def run(raw_data):\n",
" data = np.array(json.loads(raw_data)['data'])\n",
" # make prediction\n",
" y_hat = model.predict(data)\n",
" # you can return any data type as long as it is JSON-serializable\n",
" return y_hat.tolist()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create configuration file\n",
"\n",
"Create a deployment configuration file and specify the number of CPUs and gigabyte of RAM needed for your ACI container. While it depends on your model, the default of 1 core and 1 gigabyte of RAM is usually sufficient for many models. If you feel you need more later, you would have to recreate the image and redeploy the service."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"configure web service",
"aci"
]
},
"outputs": [],
"source": [
"from azureml.core.webservice import AciWebservice\n",
"\n",
"aciconfig = AciWebservice.deploy_configuration(cpu_cores=1, \n",
" memory_gb=1, \n",
" tags={\"data\": \"MNIST\", \"method\" : \"sklearn\"}, \n",
" description='Predict MNIST with sklearn')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Deploy in ACI\n",
"Estimated time to complete: **about 2-5 minutes**\n",
"\n",
"Configure the image and deploy. The following code goes through these steps:\n",
"\n",
"1. Create environment object containing dependencies needed by the model using the environment file (`myenv.yml`)\n",
"1. Create inference configuration necessary to deploy the model as a web service using:\n",
" * The scoring file (`score.py`)\n",
" * envrionment object created in previous step\n",
"1. Deploy the model to the ACI container.\n",
"1. Get the web service HTTP endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"configure image",
"create image",
"deploy web service",
"aci"
]
},
"outputs": [],
"source": [
"%%time\n",
"import uuid\n",
"from azureml.core.webservice import Webservice\n",
"from azureml.core.model import InferenceConfig\n",
"from azureml.core.environment import Environment\n",
"from azureml.core import Workspace\n",
"from azureml.core.model import Model\n",
"\n",
"ws = Workspace.from_config()\n",
"model = Model(ws, 'sklearn_mnist')\n",
"\n",
"\n",
"myenv = Environment.get(workspace=ws, name=\"tutorial-env\", version=\"1\")\n",
"inference_config = InferenceConfig(entry_script=\"score.py\", environment=myenv)\n",
"\n",
"service_name = 'sklearn-mnist-svc-' + str(uuid.uuid4())[:4]\n",
"service = Model.deploy(workspace=ws, \n",
" name=service_name, \n",
" models=[model], \n",
" inference_config=inference_config, \n",
" deployment_config=aciconfig)\n",
"\n",
"service.wait_for_deployment(show_output=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get the scoring web service's HTTP endpoint, which accepts REST client calls. This endpoint can be shared with anyone who wants to test the web service or integrate it into an application."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"get scoring uri"
]
},
"outputs": [],
"source": [
"print(service.scoring_uri)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Test the model\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Download test data\n",
"Download the test data to the **./data/** directory"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from azureml.core import Dataset\n",
"from azureml.opendatasets import MNIST\n",
"\n",
"data_folder = os.path.join(os.getcwd(), 'data')\n",
"os.makedirs(data_folder, exist_ok=True)\n",
"\n",
"mnist_file_dataset = MNIST.get_file_dataset()\n",
"mnist_file_dataset.download(data_folder, overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load test data\n",
"\n",
"Load the test data from the **./data/** directory created during the training tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from utils import load_data\n",
"import os\n",
"import glob\n",
"\n",
"data_folder = os.path.join(os.getcwd(), 'data')\n",
"# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the neural network converge faster\n",
"X_test = load_data(glob.glob(os.path.join(data_folder,\"**/t10k-images-idx3-ubyte.gz\"), recursive=True)[0], False) / 255.0\n",
"y_test = load_data(glob.glob(os.path.join(data_folder,\"**/t10k-labels-idx1-ubyte.gz\"), recursive=True)[0], True).reshape(-1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Predict test data\n",
"\n",
"Feed the test dataset to the model to get predictions.\n",
"\n",
"\n",
"The following code goes through these steps:\n",
"1. Send the data as a JSON array to the web service hosted in ACI. \n",
"\n",
"1. Use the SDK's `run` API to invoke the service. You can also make raw calls using any HTTP tool such as curl."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"test = json.dumps({\"data\": X_test.tolist()})\n",
"test = bytes(test, encoding='utf8')\n",
"y_hat = service.run(input_data=test)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Examine the confusion matrix\n",
"\n",
"Generate a confusion matrix to see how many samples from the test set are classified correctly. Notice the mis-classified value for the incorrect predictions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.metrics import confusion_matrix\n",
"\n",
"conf_mx = confusion_matrix(y_test, y_hat)\n",
"print(conf_mx)\n",
"print('Overall accuracy:', np.average(y_hat == y_test))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use `matplotlib` to display the confusion matrix as a graph. In this graph, the X axis represents the actual values, and the Y axis represents the predicted values. The color in each grid represents the error rate. The lighter the color, the higher the error rate is. For example, many 5's are mis-classified as 3's. Hence you see a bright grid at (5,3)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# normalize the diagonal cells so that they don't overpower the rest of the cells when visualized\n",
"row_sums = conf_mx.sum(axis=1, keepdims=True)\n",
"norm_conf_mx = conf_mx / row_sums\n",
"np.fill_diagonal(norm_conf_mx, 0)\n",
"\n",
"fig = plt.figure(figsize=(8,5))\n",
"ax = fig.add_subplot(111)\n",
"cax = ax.matshow(norm_conf_mx, cmap=plt.cm.bone)\n",
"ticks = np.arange(0, 10, 1)\n",
"ax.set_xticks(ticks)\n",
"ax.set_yticks(ticks)\n",
"ax.set_xticklabels(ticks)\n",
"ax.set_yticklabels(ticks)\n",
"fig.colorbar(cax)\n",
"plt.ylabel('true labels', fontsize=14)\n",
"plt.xlabel('predicted values', fontsize=14)\n",
"plt.savefig('conf.png')\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Show predictions\n",
"\n",
"Test the deployed model with a random sample of 30 images from the test data. \n",
"\n",
"\n",
"1. Print the returned predictions and plot them along with the input images. Red font and inverse image (white on black) is used to highlight the misclassified samples. \n",
"\n",
" Since the model accuracy is high, you might have to run the following code a few times before you can see a misclassified sample."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"score web service"
]
},
"outputs": [],
"source": [
"import json\n",
"\n",
"# find 30 random samples from test set\n",
"n = 30\n",
"sample_indices = np.random.permutation(X_test.shape[0])[0:n]\n",
"\n",
"test_samples = json.dumps({\"data\": X_test[sample_indices].tolist()})\n",
"test_samples = bytes(test_samples, encoding='utf8')\n",
"\n",
"# predict using the deployed model\n",
"result = service.run(input_data=test_samples)\n",
"\n",
"# compare actual value vs. the predicted values:\n",
"i = 0\n",
"plt.figure(figsize = (20, 1))\n",
"\n",
"for s in sample_indices:\n",
" plt.subplot(1, n, i + 1)\n",
" plt.axhline('')\n",
" plt.axvline('')\n",
" \n",
" # use different color for misclassified sample\n",
" font_color = 'red' if y_test[s] != result[i] else 'black'\n",
" clr_map = plt.cm.gray if y_test[s] != result[i] else plt.cm.Greys\n",
" \n",
" plt.text(x=10, y =-10, s=result[i], fontsize=18, color=font_color)\n",
" plt.imshow(X_test[s].reshape(28, 28), cmap=clr_map)\n",
" \n",
" i = i + 1\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also send raw HTTP request to test the web service."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"score web service"
]
},
"outputs": [],
"source": [
"import requests\n",
"\n",
"# send a random row from the test set to score\n",
"random_index = np.random.randint(0, len(X_test)-1)\n",
"input_data = \"{\\\"data\\\": [\" + str(list(X_test[random_index])) + \"]}\"\n",
"\n",
"headers = {'Content-Type':'application/json'}\n",
"\n",
"# for AKS deployment you'd need to the service key in the header as well\n",
"# api_key = service.get_key()\n",
"# headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)} \n",
"\n",
"resp = requests.post(service.scoring_uri, input_data, headers=headers)\n",
"\n",
"print(\"POST to url\", service.scoring_uri)\n",
"#print(\"input data:\", input_data)\n",
"print(\"label:\", y_test[random_index])\n",
"print(\"prediction:\", resp.text)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up resources\n",
"\n",
"To keep the resource group and workspace for other tutorials and exploration, you can delete only the ACI deployment using this API call:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"delete web service"
]
},
"outputs": [],
"source": [
"service.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"If you're not going to use what you've created here, delete the resources you just created with this quickstart so you don't incur any charges. In the Azure portal, select and delete your resource group. You can also keep the resource group, but delete a single workspace by displaying the workspace properties and selecting the Delete button.\n",
"\n",
"\n",
"## Next steps\n",
"\n",
"In this Azure Machine Learning tutorial, you used Python to:\n",
"\n",
"> * Set up your testing environment\n",
"> * Retrieve the model from your workspace\n",
"> * Test the model locally\n",
"> * Deploy the model to ACI\n",
"> * Test the deployed model\n",
" \n",
"You can also try out the [regression tutorial](regression-part1-data-prep.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/img-classification-part2-deploy.png)"
]
}
],
"metadata": {
"authors": [
{
"name": "shipatel"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
},
"msauthor": "sgilley",
"network_required": false
},
"nbformat": 4,
"nbformat_minor": 2
}

8
tutorials/image-classification-mnist-data/img-classification-part2-deploy.yml Normal file
View File

@@ -0,0 +1,8 @@
name: img-classification-part2-deploy
dependencies:
- pip:
- azureml-sdk
- matplotlib
- sklearn
- pandas
- azureml-opendatasets

618
tutorials/image-classification-mnist-data/img-classification-part3-deploy-encrypted.ipynb Normal file
View File

@@ -0,0 +1,618 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
"\n",
"Licensed under the MIT License."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial #3: Deploy an image classification model for encrypted inferencing in Azure Container Instance (ACI)\n",
"\n",
"This tutorial is **a new addition to the two-part series**. In the [previous tutorial](img-classification-part1-training.ipynb), you trained machine learning models and then registered a model in your workspace on the cloud. \n",
"\n",
"Now, you're ready to deploy the model as a encrypted inferencing web service in [Azure Container Instances](https://docs.microsoft.com/azure/container-instances/) (ACI). A web service is an image, in this case a Docker image, that encapsulates the scoring logic and the model itself. \n",
"\n",
"In this part of the tutorial, you use Azure Machine Learning service (Preview) to:\n",
"\n",
"> * Set up your testing environment\n",
"> * Retrieve the model from your workspace\n",
"> * Test the model locally\n",
"> * Deploy the model to ACI\n",
"> * Test the deployed model\n",
"\n",
"ACI is a great solution for testing and understanding the workflow. For scalable production deployments, consider using Azure Kubernetes Service. For more information, see [how to deploy and where](https://docs.microsoft.com/azure/machine-learning/service/how-to-deploy-and-where).\n",
"\n",
"\n",
"## Prerequisites\n",
"\n",
"Complete the model training in the [Tutorial #1: Train an image classification model with Azure Machine Learning](train-models.ipynb) notebook. \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# If you did NOT complete the tutorial, you can instead run this cell \n",
"# This will register a model and download the data needed for this tutorial\n",
"# These prerequisites are created in the training tutorial\n",
"# Feel free to skip this cell if you completed the training tutorial \n",
"\n",
"# register a model\n",
"from azureml.core import Workspace\n",
"ws = Workspace.from_config()\n",
"\n",
"from azureml.core.model import Model\n",
"\n",
"model_name = \"sklearn_mnist\"\n",
"model = Model.register(model_path=\"sklearn_mnist_model.pkl\",\n",
" model_name=model_name,\n",
" tags={\"data\": \"mnist\", \"model\": \"classification\"},\n",
" description=\"Mnist handwriting recognition\",\n",
" workspace=ws)\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup the Environment \n",
"\n",
"Add `encrypted-inference` package as a conda dependency "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.environment import Environment\n",
"from azureml.core.conda_dependencies import CondaDependencies\n",
"\n",
"# to install required packages\n",
"env = Environment('tutorial-encryption-env')\n",
"cd = CondaDependencies.create(pip_packages=['azureml-dataset-runtime[pandas,fuse]', 'azureml-defaults', 'azure-storage-blob', 'encrypted-inference==0.9'], conda_packages = ['scikit-learn==0.22.1'])\n",
"\n",
"env.python.conda_dependencies = cd\n",
"\n",
"# Register environment to re-use later\n",
"env.register(workspace = ws)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set up the environment\n",
"\n",
"Start by setting up a testing environment.\n",
"\n",
"### Import packages\n",
"\n",
"Import the Python packages needed for this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"check version"
]
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
" \n",
"import azureml.core\n",
"\n",
"# display the core SDK version number\n",
"print(\"Azure ML SDK Version: \", azureml.core.VERSION)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Install Homomorphic Encryption based library for Secure Inferencing\n",
"\n",
"Our library is based on [Microsoft SEAL](https://github.com/Microsoft/SEAL) and pubished to [PyPi.org](https://pypi.org/project/encrypted-inference) as an easy to use package "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install encrypted-inference==0.9"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy as web service\n",
"\n",
"Deploy the model as a web service hosted in ACI. \n",
"\n",
"To build the correct environment for ACI, provide the following:\n",
"* A scoring script to show how to use the model\n",
"* A configuration file to build the ACI\n",
"* The model you trained before\n",
"\n",
"### Create scoring script\n",
"\n",
"Create the scoring script, called score_encrypted.py, used by the web service call to show how to use the model.\n",
"\n",
"You must include two required functions into the scoring script:\n",
"* The `init()` function, which typically loads the model into a global object. This function is run only once when the Docker container is started. \n",
"\n",
"* The `run(input_data)` function uses the model to predict a value based on the input data. Inputs and outputs to the run typically use JSON for serialization and de-serialization, but other formats are supported. The function fetches homomorphic encryption based public keys that are uploaded by the service caller. \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile score_encrypted.py\n",
"import json\n",
"import os\n",
"import pickle\n",
"import joblib\n",
"from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, PublicAccess\n",
"from encrypted.inference.eiserver import EIServer\n",
"\n",
"def init():\n",
" global model\n",
" # AZUREML_MODEL_DIR is an environment variable created during deployment.\n",
" # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)\n",
" # For multiple models, it points to the folder containing all deployed models (./azureml-models)\n",
" model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_mnist_model.pkl')\n",
" model = joblib.load(model_path)\n",
"\n",
" global server\n",
" server = EIServer(model.coef_, model.intercept_, verbose=True)\n",
"\n",
"def run(raw_data):\n",
"\n",
" json_properties = json.loads(raw_data)\n",
"\n",
" key_id = json_properties['key_id']\n",
" conn_str = json_properties['conn_str']\n",
" container = json_properties['container']\n",
" data = json_properties['data']\n",
"\n",
" # download the Galois keys from blob storage\n",
" #TODO optimize by caching the keys locally \n",
" blob_service_client = BlobServiceClient.from_connection_string(conn_str=conn_str)\n",
" blob_client = blob_service_client.get_blob_client(container=container, blob=key_id)\n",
" public_keys = blob_client.download_blob().readall()\n",
" \n",
" result = {}\n",
" # make prediction\n",
" result = server.predict(data, public_keys)\n",
"\n",
" # you can return any data type as long as it is JSON-serializable\n",
" return result"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create configuration file\n",
"\n",
"Create a deployment configuration file and specify the number of CPUs and gigabyte of RAM needed for your ACI container. While it depends on your model, the default of 1 core and 1 gigabyte of RAM is usually sufficient for many models. If you feel you need more later, you would have to recreate the image and redeploy the service."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"configure web service",
"aci"
]
},
"outputs": [],
"source": [
"from azureml.core.webservice import AciWebservice\n",
"\n",
"aciconfig = AciWebservice.deploy_configuration(cpu_cores=1, \n",
" memory_gb=1, \n",
" tags={\"data\": \"MNIST\", \"method\" : \"sklearn\"}, \n",
" description='Encrypted Predict MNIST with sklearn + SEAL')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Deploy in ACI\n",
"Estimated time to complete: **about 2-5 minutes**\n",
"\n",
"Configure the image and deploy. The following code goes through these steps:\n",
"\n",
"1. Create environment object containing dependencies needed by the model using the environment file (`myenv.yml`)\n",
"1. Create inference configuration necessary to deploy the model as a web service using:\n",
" * The scoring file (`score_encrypted.py`)\n",
" * envrionment object created in previous step\n",
"1. Deploy the model to the ACI container.\n",
"1. Get the web service HTTP endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"configure image",
"create image",
"deploy web service",
"aci"
]
},
"outputs": [],
"source": [
"%%time\n",
"import uuid\n",
"from azureml.core.webservice import Webservice\n",
"from azureml.core.model import InferenceConfig\n",
"from azureml.core.environment import Environment\n",
"from azureml.core import Workspace\n",
"from azureml.core.model import Model\n",
"\n",
"ws = Workspace.from_config()\n",
"model = Model(ws, 'sklearn_mnist')\n",
"\n",
"myenv = Environment.get(workspace=ws, name=\"tutorial-encryption-env\")\n",
"inference_config = InferenceConfig(entry_script=\"score_encrypted.py\", environment=myenv)\n",
"\n",
"service_name = 'sklearn-mnist-svc-' + str(uuid.uuid4())[:4]\n",
"service = Model.deploy(workspace=ws, \n",
" name=service_name, \n",
" models=[model], \n",
" inference_config=inference_config, \n",
" deployment_config=aciconfig)\n",
"\n",
"service.wait_for_deployment(show_output=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get the scoring web service's HTTP endpoint, which accepts REST client calls. This endpoint can be shared with anyone who wants to test the web service or integrate it into an application."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"get scoring uri"
]
},
"outputs": [],
"source": [
"print(service.scoring_uri)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Test the model\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Download test data\n",
"Download the test data to the **./data/** directory"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from azureml.core import Dataset\n",
"from azureml.opendatasets import MNIST\n",
"\n",
"data_folder = os.path.join(os.getcwd(), 'data')\n",
"os.makedirs(data_folder, exist_ok=True)\n",
"\n",
"mnist_file_dataset = MNIST.get_file_dataset()\n",
"mnist_file_dataset.download(data_folder, overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load test data\n",
"\n",
"Load the test data from the **./data/** directory created during the training tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from utils import load_data\n",
"import os\n",
"import glob\n",
"\n",
"data_folder = os.path.join(os.getcwd(), 'data')\n",
"# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the neural network converge faster\n",
"X_test = load_data(glob.glob(os.path.join(data_folder,\"**/t10k-images-idx3-ubyte.gz\"), recursive=True)[0], False) / 255.0\n",
"y_test = load_data(glob.glob(os.path.join(data_folder,\"**/t10k-labels-idx1-ubyte.gz\"), recursive=True)[0], True).reshape(-1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Predict test data\n",
"\n",
"Feed the test dataset to the model to get predictions.\n",
"\n",
"\n",
"The following code goes through these steps:\n",
"\n",
"1. Create our Homomorphic Encryption based client \n",
"\n",
"1. Upload HE generated public keys \n",
"\n",
"1. Encrypt the data\n",
"\n",
"1. Send the data as JSON to the web service hosted in ACI. \n",
"\n",
"1. Use the SDK's `run` API to invoke the service. You can also make raw calls using any HTTP tool such as curl."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Create our Homomorphic Encryption based client \n",
"\n",
"Create a new EILinearRegressionClient and setup the public keys "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from encrypted.inference.eiclient import EILinearRegressionClient\n",
"\n",
"# Create a new Encrypted inference client and a new secret key.\n",
"edp = EILinearRegressionClient(verbose=True)\n",
"\n",
"public_keys_blob, public_keys_data = edp.get_public_keys()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Upload HE generated public keys\n",
"\n",
"Upload the public keys to the workspace default blob store. This will allow us to share the keys with the inference server"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import azureml.core\n",
"from azureml.core import Workspace, Datastore\n",
"import os\n",
"\n",
"ws = Workspace.from_config()\n",
"\n",
"datastore = ws.get_default_datastore()\n",
"container_name=datastore.container_name\n",
"\n",
"# Create a local file and write the keys to it\n",
"public_keys = open(public_keys_blob, \"wb\")\n",
"public_keys.write(public_keys_data)\n",
"public_keys.close()\n",
"\n",
"# Upload the file to blob store\n",
"datastore.upload_files([public_keys_blob])\n",
"\n",
"# Delete the local file\n",
"os.remove(public_keys_blob)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Encrypt the data "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#choose any one sample from the test data \n",
"sample_index = 1\n",
"\n",
"#encrypt the data\n",
"raw_data = edp.encrypt(X_test[sample_index])\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Send the test data to the webservice hosted in ACI\n",
"\n",
"Feed the test dataset to the model to get predictions. We will need to send the connection string to the blob storage where the public keys were uploaded \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"from azureml.core import Webservice\n",
"\n",
"service = Webservice(ws, service_name)\n",
"\n",
"#pass the connection string for blob storage to give the server access to the uploaded public keys \n",
"conn_str_template = 'DefaultEndpointsProtocol={};AccountName={};AccountKey={};EndpointSuffix=core.windows.net'\n",
"conn_str = conn_str_template.format(datastore.protocol, datastore.account_name, datastore.account_key)\n",
"\n",
"#build the json \n",
"data = json.dumps({\"data\": raw_data, \"key_id\" : public_keys_blob, \"conn_str\" : conn_str, \"container\" : container_name })\n",
"data = bytes(data, encoding='ASCII')\n",
"\n",
"print ('Making an encrypted inference web service call ')\n",
"eresult = service.run(input_data=data)\n",
"\n",
"print ('Received encrypted inference results')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Decrypt the data\n",
"\n",
"Use the client to decrypt the results"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np \n",
"\n",
"results = edp.decrypt(eresult)\n",
"\n",
"print ('Decrypted the results ', results)\n",
"\n",
"#Apply argmax to identify the prediction result\n",
"prediction = np.argmax(results)\n",
"\n",
"print ( ' Prediction : ', prediction)\n",
"print ( ' Actual Label : ', y_test[sample_index])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up resources\n",
"\n",
"To keep the resource group and workspace for other tutorials and exploration, you can delete only the ACI deployment using this API call:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"delete web service"
]
},
"outputs": [],
"source": [
"service.delete()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"If you're not going to use what you've created here, delete the resources you just created with this quickstart so you don't incur any charges. In the Azure portal, select and delete your resource group. You can also keep the resource group, but delete a single workspace by displaying the workspace properties and selecting the Delete button.\n",
"\n",
"\n",
"## Next steps\n",
"\n",
"In this Azure Machine Learning tutorial, you used Python to:\n",
"\n",
"> * Set up your testing environment\n",
"> * Retrieve the model from your workspace\n",
"> * Test the model locally\n",
"> * Deploy the model to ACI\n",
"> * Test the deployed model\n",
" \n",
"You can also try out the [regression tutorial](regression-part1-data-prep.ipynb)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/img-classification-part2-deploy.png)"
]
}
],
"metadata": {
"authors": [
{
"name": "vkanne"
}
],
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6"
},
"msauthor": "vkanne",
"network_required": false
},
"nbformat": 4,
"nbformat_minor": 2
}

10
tutorials/image-classification-mnist-data/img-classification-part3-deploy-encrypted.yml Normal file
View File

@@ -0,0 +1,10 @@
name: img-classification-part3-deploy-encrypted
dependencies:
- pip:
- azureml-sdk
- matplotlib
- sklearn
- pandas
- azureml-opendatasets
- encrypted-inference==0.9
- azure-storage-blob

BIN
tutorials/image-classification-mnist-data/sklearn_mnist_model.pkl Normal file
View File

Binary file not shown.

27
tutorials/image-classification-mnist-data/utils.py Normal file
View File

@@ -0,0 +1,27 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
import gzip
import numpy as np
import struct
# load compressed MNIST gz files and return numpy arrays
def load_data(filename, label=False):
with gzip.open(filename) as gz:
struct.unpack('I', gz.read(4))
n_items = struct.unpack('>I', gz.read(4))
if not label:
n_rows = struct.unpack('>I', gz.read(4))[0]
n_cols = struct.unpack('>I', gz.read(4))[0]
res = np.frombuffer(gz.read(n_items[0] * n_rows * n_cols), dtype=np.uint8)
res = res.reshape(n_items[0], n_rows * n_cols)
else:
res = np.frombuffer(gz.read(n_items[0]), dtype=np.uint8)
res = res.reshape(n_items[0], 1)
return res
# one-hot encode a 1-D array
def one_hot_encode(array, num_of_classes):
return np.eye(num_of_classes)[array.reshape(-1)]

80
tutorials/machine-learning-pipelines-advanced/scripts/batch_scoring.py Normal file
View File

@@ -0,0 +1,80 @@
# Copyright (c) Microsoft. All rights reserved.
# Licensed under the MIT license.
import os
import argparse
import datetime
import time
import tensorflow as tf
from math import ceil
import numpy as np
import shutil
from tensorflow.contrib.slim.python.slim.nets import inception_v3
from azureml.core import Run
from azureml.core.model import Model
from azureml.core.dataset import Dataset
slim = tf.contrib.slim
image_size = 299
num_channel = 3
def get_class_label_dict(labels_dir):
label = []
labels_path = os.path.join(labels_dir, 'labels.txt')
proto_as_ascii_lines = tf.gfile.GFile(labels_path).readlines()
for temp in proto_as_ascii_lines:
label.append(temp.rstrip())
return label
def init():
global g_tf_sess, probabilities, label_dict, input_images
parser = argparse.ArgumentParser(description="Start a tensorflow model serving")
parser.add_argument('--model_name', dest="model_name", required=True)
parser.add_argument('--labels_dir', dest="labels_dir", required=True)
args, _ = parser.parse_known_args()
label_dict = get_class_label_dict(args.labels_dir)
classes_num = len(label_dict)
with slim.arg_scope(inception_v3.inception_v3_arg_scope()):
input_images = tf.placeholder(tf.float32, [1, image_size, image_size, num_channel])
logits, _ = inception_v3.inception_v3(input_images,
num_classes=classes_num,
is_training=False)
probabilities = tf.argmax(logits, 1)
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
g_tf_sess = tf.Session(config=config)
g_tf_sess.run(tf.global_variables_initializer())
g_tf_sess.run(tf.local_variables_initializer())
model_path = Model.get_model_path(args.model_name)
saver = tf.train.Saver()
saver.restore(g_tf_sess, model_path)
def file_to_tensor(file_path):
image_string = tf.read_file(file_path)
image = tf.image.decode_image(image_string, channels=3)
image.set_shape([None, None, None])
image = tf.image.resize_images(image, [image_size, image_size])
image = tf.divide(tf.subtract(image, [0]), [255])
image.set_shape([image_size, image_size, num_channel])
return image
def run(mini_batch):
result_list = []
for file_path in mini_batch:
test_image = file_to_tensor(file_path)
out = g_tf_sess.run(test_image)
result = g_tf_sess.run(probabilities, feed_dict={input_images: [out]})
result_list.append(os.path.basename(file_path) + ": " + label_dict[result[0]])
return result_list

677
tutorials/machine-learning-pipelines-advanced/tutorial-pipeline-batch-scoring-classification.ipynb Normal file
View File

@@ -0,0 +1,677 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved. \n",
"Licensed under the MIT License."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/machine-learning-pipelines-advanced/tutorial-pipeline-batch-scoring-classification.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Use Azure Machine Learning Pipelines for batch prediction\n",
"In this tutorial, you use Azure Machine Learning service pipelines to run a batch scoring image classification job. The example job uses the pre-trained [Inception-V3](https://arxiv.org/abs/1512.00567) CNN (convolutional neural network) Tensorflow model to classify unlabeled images. Machine learning pipelines optimize your workflow with speed, portability, and reuse so you can focus on your expertise, machine learning, rather than on infrastructure and automation. After building and publishing a pipeline, you can configure a REST endpoint to enable triggering the pipeline from any HTTP library on any platform.\n",
"\n",
"In this tutorial, you learn the following tasks:\n",
"\n",
"> * Configure workspace and download sample data\n",
"> * Create data objects to fetch and output data\n",
"> * Download, prepare, and register the model to your workspace\n",
"> * Provision compute targets and create a scoring script\n",
"> * Use ParallelRunStep to do batch scoring\n",
"> * Build, run, and publish a pipeline\n",
"> * Enable a REST endpoint for the pipeline\n",
"\n",
"If you don't have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning service](https://aka.ms/AMLFree) today."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites\n",
"\n",
"* Complete the [setup tutorial](https://docs.microsoft.com/azure/machine-learning/service/tutorial-1st-experiment-sdk-setup) if you don't already have an Azure Machine Learning service workspace or notebook virtual machine.\n",
"* After you complete the setup tutorial, open the **tutorials/tutorial-pipeline-batch-scoring-classification.ipynb** notebook using the same notebook server.\n",
"\n",
"This tutorial is also available on [GitHub](https://github.com/Azure/MachineLearningNotebooks/tree/master/tutorials) if you wish to run it in your own [local environment](how-to-configure-environment.md#local). Run `pip install azureml-sdk[notebooks] azureml-pipeline-core azureml-pipeline-steps pandas requests` to get the required packages."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configure workspace and create datastore"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a workspace object from the existing workspace. A [Workspace](https://docs.microsoft.com/python/api/azureml-core/azureml.core.workspace.workspace?view=azure-ml-py) is a class that accepts your Azure subscription and resource information. It also creates a cloud resource to monitor and track your model runs. `Workspace.from_config()` reads the file **config.json** and loads the authentication details into an object named `ws`. `ws` is used throughout the rest of the code in this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Check core SDK version number\n",
"import azureml.core\n",
"\n",
"print(\"SDK version:\", azureml.core.VERSION)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from azureml.core import Workspace\n",
"\n",
"ws = Workspace.from_config()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create a datastore for sample images\n",
"\n",
"Get the ImageNet evaluation public data sample from the public blob container `sampledata` on the account `pipelinedata`. Calling `register_azure_blob_container()` makes the data available to the workspace under the name `images_datastore`. Then specify the workspace default datastore as the output datastore, which you use for scoring output in the pipeline."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.datastore import Datastore\n",
"\n",
"batchscore_blob = Datastore.register_azure_blob_container(ws, \n",
" datastore_name=\"images_datastore\", \n",
" container_name=\"sampledata\", \n",
" account_name=\"pipelinedata\", \n",
" overwrite=True)\n",
"\n",
"def_data_store = ws.get_default_datastore()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create data objects\n",
"\n",
"When building pipelines, `Dataset` objects are used for reading data from workspace datastores, and `PipelineData` objects are used for transferring intermediate data between pipeline steps.\n",
"\n",
"This batch scoring example only uses one pipeline step, but in use-cases with multiple steps, the typical flow will include:\n",
"\n",
"1. Using `Dataset` objects as **inputs** to fetch raw data, performing some transformations, then **outputting** a `PipelineData` object.\n",
"1. Use the previous step's `PipelineData` **output object** as an *input object*, repeated for subsequent steps.\n",
"\n",
"For this scenario you create `Dataset` objects corresponding to the datastore directories for both the input images and the classification labels (y-test values). You also create a `PipelineData` object for the batch scoring output data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.dataset import Dataset\n",
"from azureml.pipeline.core import PipelineData\n",
"\n",
"input_images = Dataset.File.from_files((batchscore_blob, \"batchscoring/images/\"))\n",
"label_ds = Dataset.File.from_files((batchscore_blob, \"batchscoring/labels/\"))\n",
"output_dir = PipelineData(name=\"scores\", datastore=def_data_store)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we need to register the datasets with the workspace."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"input_images = input_images.register(workspace=ws, name=\"input_images\")\n",
"label_ds = label_ds.register(workspace=ws, name=\"label_ds\", create_new_version=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download and register the model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Download the pre-trained Tensorflow model to use it for batch scoring in the pipeline. First create a local directory where you store the model, then download and extract it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import tarfile\n",
"import urllib.request\n",
"\n",
"if not os.path.isdir(\"models\"):\n",
" os.mkdir(\"models\")\n",
" \n",
"response = urllib.request.urlretrieve(\"http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz\", \"model.tar.gz\")\n",
"tar = tarfile.open(\"model.tar.gz\", \"r:gz\")\n",
"tar.extractall(\"models\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now you register the model to your workspace, which allows you to easily retrieve it in the pipeline process. In the `register()` static function, the `model_name` parameter is the key you use to locate your model throughout the SDK."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import shutil\n",
"from azureml.core.model import Model\n",
"\n",
"# register downloaded model \n",
"model = Model.register(model_path=\"models/inception_v3.ckpt\",\n",
" model_name=\"inception\",\n",
" tags={\"pretrained\": \"inception\"},\n",
" description=\"Imagenet trained tensorflow inception\",\n",
" workspace=ws)\n",
"# remove the downloaded dir after registration if you wish\n",
"shutil.rmtree(\"models\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create and attach remote compute target\n",
"\n",
"Azure Machine Learning service pipelines cannot be run locally, and only run on cloud resources. Remote compute targets are reusable virtual compute environments where you run experiments and work-flows. Run the following code to create a GPU-enabled [`AmlCompute`](https://docs.microsoft.com/python/api/azureml-core/azureml.core.compute.amlcompute.amlcompute?view=azure-ml-py) target, and attach it to your workspace. See the [conceptual article](https://docs.microsoft.com/azure/machine-learning/service/concept-compute-target) for more information on compute targets.\n",
"\n",
"> Note that if you have an AzureML Data Scientist role, you will not have permission to create compute resources. Talk to your workspace or IT admin to create the compute targets described in this section, if they do not already exist."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.compute import AmlCompute, ComputeTarget\n",
"from azureml.exceptions import ComputeTargetException\n",
"compute_name = \"gpu-cluster\"\n",
"\n",
"# checks to see if compute target already exists in workspace, else create it\n",
"try:\n",
" compute_target = ComputeTarget(workspace=ws, name=compute_name)\n",
"except ComputeTargetException:\n",
" config = AmlCompute.provisioning_configuration(vm_size=\"STANDARD_NC6\",\n",
" vm_priority=\"lowpriority\", \n",
" min_nodes=0, \n",
" max_nodes=1)\n",
"\n",
" compute_target = ComputeTarget.create(workspace=ws, name=compute_name, provisioning_configuration=config)\n",
" compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Write a scoring script"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To do the scoring, you create a batch scoring script `batch_scoring.py`, and write it to the current directory. The script takes a minibatch of input images, applies the classification model, and outputs the predictions to a results file.\n",
"\n",
"The script `batch_scoring.py` takes the following parameters, which get passed from the `ParallelRunStep` that you create later:\n",
"\n",
"- `--model_name`: the name of the model being used\n",
"- `--labels_dir` : the directory path having the `labels.txt` file \n",
"\n",
"The pipelines infrastructure uses the `ArgumentParser` class to pass parameters into pipeline steps. For example, in the code below the first argument `--model_name` is given the property identifier `model_name`. In the `main()` function, this property is accessed using `Model.get_model_path(args.model_name)`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The pipeline in this tutorial only has one step and writes the output to a file, but for multi-step pipelines, you also use `ArgumentParser` to define a directory to write output data for input to subsequent steps. See the [notebook](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/nyc-taxi-data-regression-model-building/nyc-taxi-data-regression-model-building.ipynb) for an example of passing data between multiple pipeline steps using the `ArgumentParser` design pattern."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Build and run the pipeline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before running the pipeline, you create an object that defines the python environment and dependencies needed by your script `batch_scoring.py`. The main dependency required is Tensorflow, but you also install `azureml-core` and `azureml-dataset-runtime[fuse]` for background processes from the SDK. Create a `RunConfiguration` object using the dependencies, and also specify Docker and Docker-GPU support."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core import Environment\n",
"from azureml.core.conda_dependencies import CondaDependencies\n",
"from azureml.core.runconfig import DEFAULT_GPU_IMAGE\n",
"\n",
"cd = CondaDependencies.create(pip_packages=[\"tensorflow-gpu==1.15.2\",\n",
" \"azureml-core\", \"azureml-dataset-runtime[fuse]\"])\n",
"\n",
"env = Environment(name=\"parallelenv\")\n",
"env.python.conda_dependencies=cd\n",
"env.docker.base_image = DEFAULT_GPU_IMAGE"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create the configuration to wrap the inference script\n",
"Create the pipeline step using the script, environment configuration, and parameters. Specify the compute target you already attached to your workspace as the target of execution of the script. We will use PythonScriptStep to create the pipeline step."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.pipeline.steps import ParallelRunConfig\n",
"\n",
"parallel_run_config = ParallelRunConfig(\n",
" environment=env,\n",
" entry_script=\"batch_scoring.py\",\n",
" source_directory=\"scripts\",\n",
" output_action=\"append_row\",\n",
" append_row_file_name=\"parallel_run_step.txt\",\n",
" mini_batch_size=\"20\",\n",
" error_threshold=1,\n",
" compute_target=compute_target,\n",
" process_count_per_node=2,\n",
" node_count=2\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Create the pipeline step\n",
"\n",
"A pipeline step is an object that encapsulates everything you need for running a pipeline including:\n",
"\n",
"* environment and dependency settings\n",
"* the compute resource to run the pipeline on\n",
"* input and output data, and any custom parameters\n",
"* reference to a script or SDK-logic to run during the step\n",
"\n",
"There are multiple classes that inherit from the parent class [`PipelineStep`](https://docs.microsoft.com/python/api/azureml-pipeline-steps/azureml.pipeline.steps.parallelrunstep?view=azure-ml-py) to assist with building a step using certain frameworks and stacks. In this example, you use the [`ParallelRunStep`](https://docs.microsoft.com/en-us/python/api/azureml-contrib-pipeline-steps/azureml.contrib.pipeline.steps.parallelrunstep?view=azure-ml-py) class to define your step logic using a scoring script. \n",
"\n",
"An object reference in the `outputs` array becomes available as an **input** for a subsequent pipeline step, for scenarios where there is more than one step."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.pipeline.steps import ParallelRunStep\n",
"from datetime import datetime\n",
"import uuid\n",
"\n",
"parallel_step_name = \"batchscoring-\" + datetime.now().strftime(\"%Y%m%d%H%M\")\n",
"\n",
"label_config = label_ds.as_named_input(\"labels_input\").as_mount(\"/tmp/{}\".format(str(uuid.uuid4())))\n",
"\n",
"batch_score_step = ParallelRunStep(\n",
" name=parallel_step_name,\n",
" inputs=[input_images.as_named_input(\"input_images\")],\n",
" output=output_dir,\n",
" arguments=[\"--model_name\", \"inception\",\n",
" \"--labels_dir\", label_config],\n",
" side_inputs=[label_config],\n",
" parallel_run_config=parallel_run_config,\n",
" allow_reuse=False\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For a list of all classes for different step types, see the [steps package](https://docs.microsoft.com/python/api/azureml-pipeline-steps/azureml.pipeline.steps?view=azure-ml-py)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Run the pipeline\n",
"\n",
"Now you run the pipeline. First create a `Pipeline` object with your workspace reference and the pipeline step you created. The `steps` parameter is an array of steps, and in this case there is only one step for batch scoring. To build pipelines with multiple steps, you place the steps in order in this array.\n",
"\n",
"Next use the `Experiment.submit()` function to submit the pipeline for execution. You also specify the custom parameter `param_batch_size`. The `wait_for_completion` function will output logs during the pipeline build process, which allows you to see current progress.\n",
"\n",
"Note: The first pipeline run takes roughly **15 minutes**, as all dependencies must be downloaded, a Docker image is created, and the Python environment is provisioned/created. Running it again takes significantly less time as those resources are reused. However, total run time depends on the workload of your scripts and processes running in each pipeline step."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core import Experiment\n",
"from azureml.pipeline.core import Pipeline\n",
"\n",
"pipeline = Pipeline(workspace=ws, steps=[batch_score_step])\n",
"pipeline_run = Experiment(ws, \"Tutorial-Batch-Scoring\").submit(pipeline)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# This will output information of the pipeline run, including the link to the details page of portal.\n",
"pipeline_run"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Wait the run for completion and show output log to console\n",
"pipeline_run.wait_for_completion(show_output=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Download and review output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the following code to download the output file created from the `batch_scoring.py` script, then explore the scoring results."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import tempfile\n",
"\n",
"batch_run = pipeline_run.find_step_run(batch_score_step.name)[0]\n",
"batch_output = batch_run.get_output_data(output_dir.name)\n",
"\n",
"target_dir = tempfile.mkdtemp()\n",
"batch_output.download(local_path=target_dir)\n",
"result_file = os.path.join(target_dir, batch_output.path_on_datastore, parallel_run_config.append_row_file_name)\n",
"\n",
"df = pd.read_csv(result_file, delimiter=\":\", header=None)\n",
"df.columns = [\"Filename\", \"Prediction\"]\n",
"print(\"Prediction has \", df.shape[0], \" rows\")\n",
"df.head(10) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Publish and run from REST endpoint"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the following code to publish the pipeline to your workspace. In your workspace in the portal, you can see metadata for the pipeline including run history and durations. You can also run the pipeline manually from the portal.\n",
"\n",
"Additionally, publishing the pipeline enables a REST endpoint to rerun the pipeline from any HTTP library on any platform."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"published_pipeline = pipeline_run.publish_pipeline(\n",
" name=\"Inception_v3_scoring\", description=\"Batch scoring using Inception v3 model\", version=\"1.0\")\n",
"\n",
"published_pipeline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To run the pipeline from the REST endpoint, you first need an OAuth2 Bearer-type authentication header. This example uses interactive authentication for illustration purposes, but for most production scenarios requiring automated or headless authentication, use service principle authentication as [described in this notebook](https://aka.ms/pl-restep-auth).\n",
"\n",
"Service principle authentication involves creating an **App Registration** in **Azure Active Directory**, generating a client secret, and then granting your service principal **role access** to your machine learning workspace. You then use the [`ServicePrincipalAuthentication`](https://docs.microsoft.com/python/api/azureml-core/azureml.core.authentication.serviceprincipalauthentication?view=azure-ml-py) class to manage your auth flow. \n",
"\n",
"Both `InteractiveLoginAuthentication` and `ServicePrincipalAuthentication` inherit from `AbstractAuthentication`, and in both cases you use the `get_authentication_header()` function in the same way to fetch the header."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.authentication import InteractiveLoginAuthentication\n",
"\n",
"interactive_auth = InteractiveLoginAuthentication()\n",
"auth_header = interactive_auth.get_authentication_header()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get the REST url from the `endpoint` property of the published pipeline object. You can also find the REST url in your workspace in the portal. Build an HTTP POST request to the endpoint, specifying your authentication header. Additionally, add a JSON payload object with the experiment name and the batch size parameter. As a reminder, the `process_count_per_node` is passed through to `ParallelRunStep` because you defined it is defined as a `PipelineParameter` object in the step configuration.\n",
"\n",
"Make the request to trigger the run. Access the `Id` key from the response dict to get the value of the run id."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"\n",
"rest_endpoint = published_pipeline.endpoint\n",
"response = requests.post(rest_endpoint, \n",
" headers=auth_header, \n",
" json={\"ExperimentName\": \"Tutorial-Batch-Scoring\",\n",
" \"ParameterAssignments\": {\"process_count_per_node\": 6}})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"try:\n",
" response.raise_for_status()\n",
"except Exception: \n",
" raise Exception(\"Received bad response from the endpoint: {}\\n\"\n",
" \"Response Code: {}\\n\"\n",
" \"Headers: {}\\n\"\n",
" \"Content: {}\".format(rest_endpoint, response.status_code, response.headers, response.content))\n",
"\n",
"run_id = response.json().get('Id')\n",
"print('Submitted pipeline run: ', run_id)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the run id to monitor the status of the new run. This will take another 10-15 min to run and will look similar to the previous pipeline run, so if you don't need to see another pipeline run, you can skip watching the full output."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.pipeline.core.run import PipelineRun\n",
"\n",
"published_pipeline_run = PipelineRun(ws.experiments[\"Tutorial-Batch-Scoring\"], run_id)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Show detail information of the run\n",
"published_pipeline_run"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up resources\n",
"\n",
"Do not complete this section if you plan on running other Azure Machine Learning service tutorials.\n",
"\n",
"### Stop the notebook VM\n",
"\n",
"If you used a cloud notebook server, stop the VM when you are not using it to reduce cost.\n",
"\n",
"1. In your workspace, select **Compute**.\n",
"1. Select the **Notebook VMs** tab in the compute page.\n",
"1. From the list, select the VM.\n",
"1. Select **Stop**.\n",
"1. When you're ready to use the server again, select **Start**.\n",
"\n",
"### Delete everything\n",
"\n",
"If you don't plan to use the resources you created, delete them, so you don't incur any charges.\n",
"\n",
"1. In the Azure portal, select **Resource groups** on the far left.\n",
"1. From the list, select the resource group you created.\n",
"1. Select **Delete resource group**.\n",
"1. Enter the resource group name. Then select **Delete**.\n",
"\n",
"You can also keep the resource group but delete a single workspace. Display the workspace properties and select **Delete**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Next steps\n",
"\n",
"In this machine learning pipelines tutorial, you did the following tasks:\n",
"\n",
"> * Built a pipeline with environment dependencies to run on a remote GPU compute resource\n",
"> * Created a scoring script to run batch predictions with a pre-trained Tensorflow model\n",
"> * Published a pipeline and enabled it to be run from a REST endpoint\n",
"\n",
"See the [how-to](https://docs.microsoft.com/azure/machine-learning/service/how-to-create-your-first-pipeline?view=azure-devops) for additional detail on building pipelines with the machine learning SDK."
]
}
],
"metadata": {
"authors": [
{
"name": [
"sanpil",
"trmccorm",
"pansav"
]
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
},
"msauthor": "trbye"
},
"nbformat": 4,
"nbformat_minor": 2
}

9
tutorials/machine-learning-pipelines-advanced/tutorial-pipeline-batch-scoring-classification.yml Normal file
View File

@@ -0,0 +1,9 @@
name: tutorial-pipeline-batch-scoring-classification
dependencies:
- pip:
- azureml-sdk
- azureml-pipeline-core
- azureml-pipeline-steps
- pandas
- requests
- azureml-widgets

615
tutorials/regression-automl-nyc-taxi-data/regression-automated-ml.ipynb Normal file
View File

@@ -0,0 +1,615 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright (c) Microsoft Corporation. All rights reserved."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/tutorials/regression-part2-automated-ml.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tutorial: Use automated machine learning to predict taxi fares"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this tutorial, you use automated machine learning in Azure Machine Learning service to create a regression model to predict NYC taxi fare prices. This process accepts training data and configuration settings, and automatically iterates through combinations of different feature normalization/standardization methods, models, and hyperparameter settings to arrive at the best model.\n",
"\n",
"In this tutorial you learn the following tasks:\n",
"\n",
"* Download, transform, and clean data using Azure Open Datasets\n",
"* Train an automated machine learning regression model\n",
"* Calculate model accuracy\n",
"\n",
"If you don\u00e2\u20ac\u2122t have an Azure subscription, create a free account before you begin. Try the [free or paid version](https://aka.ms/AMLFree) of Azure Machine Learning service today."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prerequisites"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* Complete the [setup tutorial](https://docs.microsoft.com/azure/machine-learning/service/tutorial-1st-experiment-sdk-setup) if you don't already have an Azure Machine Learning service workspace or notebook virtual machine.\n",
"* After you complete the setup tutorial, open the **tutorials/regression-automated-ml.ipynb** notebook using the same notebook server.\n",
"\n",
"This tutorial is also available on [GitHub](https://github.com/Azure/MachineLearningNotebooks/tree/master/tutorials) if you wish to run it in your own [local environment](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/automated-machine-learning/README.md#setup-using-a-local-conda-environment)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download and prepare data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"from azureml.core import Dataset\n",
"from datetime import datetime\n",
"from dateutil.relativedelta import relativedelta"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Begin by creating a dataframe to hold the taxi data. Then preview the data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"green_taxi_dataset = Dataset.Tabular.from_parquet_files(path=\"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/green_taxi_data.parquet\")\n",
"green_taxi_df = green_taxi_dataset.to_pandas_dataframe()\n",
"green_taxi_df.head(10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remove some of the columns that you won't need for training or additional feature building. Automate machine learning will automatically handle time-based features such as lpepPickupDatetime."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"columns_to_remove = [\"lpepDropoffDatetime\", \"puLocationId\", \"doLocationId\", \"extra\", \"mtaTax\",\n",
" \"improvementSurcharge\", \"tollsAmount\", \"ehailFee\", \"tripType\", \"rateCodeID\", \n",
" \"storeAndFwdFlag\", \"paymentType\", \"fareAmount\", \"tipAmount\"\n",
" ]\n",
"for col in columns_to_remove:\n",
" green_taxi_df.pop(col)\n",
" \n",
"green_taxi_df.head(5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Cleanse data "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the `describe()` function on the new dataframe to see summary statistics for each field."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"green_taxi_df.describe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"From the summary statistics, you see that there are several fields that have outliers or values that will reduce model accuracy. First filter the lat/long fields to be within the bounds of the Manhattan area. This will filter out longer taxi trips or trips that are outliers in respect to their relationship with other features. \n",
"\n",
"Additionally filter the `tripDistance` field to be greater than zero but less than 31 miles (the haversine distance between the two lat/long pairs). This eliminates long outlier trips that have inconsistent trip cost.\n",
"\n",
"Lastly, the `totalAmount` field has negative values for the taxi fares, which don't make sense in the context of our model, and the `passengerCount` field has bad data with the minimum values being zero.\n",
"\n",
"Filter out these anomalies using query functions, and then remove the last few columns unnecessary for training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"final_df = green_taxi_df.query(\"pickupLatitude>=40.53 and pickupLatitude<=40.88\")\n",
"final_df = final_df.query(\"pickupLongitude>=-74.09 and pickupLongitude<=-73.72\")\n",
"final_df = final_df.query(\"tripDistance>=0.25 and tripDistance<31\")\n",
"final_df = final_df.query(\"passengerCount>0 and totalAmount>0\")\n",
"\n",
"columns_to_remove_for_training = [\"pickupLongitude\", \"pickupLatitude\", \"dropoffLongitude\", \"dropoffLatitude\"]\n",
"for col in columns_to_remove_for_training:\n",
" final_df.pop(col)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Call `describe()` again on the data to ensure cleansing worked as expected. You now have a prepared and cleansed set of taxi, holiday, and weather data to use for machine learning model training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"final_df.describe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configure workspace\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a workspace object from the existing workspace. A [Workspace](https://docs.microsoft.com/python/api/azureml-core/azureml.core.workspace.workspace?view=azure-ml-py) is a class that accepts your Azure subscription and resource information. It also creates a cloud resource to monitor and track your model runs. `Workspace.from_config()` reads the file **config.json** and loads the authentication details into an object named `ws`. `ws` is used throughout the rest of the code in this tutorial."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.workspace import Workspace\n",
"ws = Workspace.from_config()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Split the data into train and test sets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Split the data into training and test sets by using the `train_test_split` function in the `scikit-learn` library. This function segregates the data into the x (**features**) data set for model training and the y (**values to predict**) data set for testing. The `test_size` parameter determines the percentage of data to allocate to testing. The `random_state` parameter sets a seed to the random generator, so that your train-test splits are deterministic."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.model_selection import train_test_split\n",
"\n",
"x_train, x_test = train_test_split(final_df, test_size=0.2, random_state=223)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The purpose of this step is to have data points to test the finished model that haven't been used to train the model, in order to measure true accuracy. \n",
"\n",
"In other words, a well-trained model should be able to accurately make predictions from data it hasn't already seen. You now have data prepared for auto-training a machine learning model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Automatically train a model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To automatically train a model, take the following steps:\n",
"1. Define settings for the experiment run. Attach your training data to the configuration, and modify settings that control the training process.\n",
"1. Submit the experiment for model tuning. After submitting the experiment, the process iterates through different machine learning algorithms and hyperparameter settings, adhering to your defined constraints. It chooses the best-fit model by optimizing an accuracy metric."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Define training settings"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define the experiment parameter and model settings for training. View the full list of [settings](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-auto-train). Submitting the experiment with these default settings will take approximately 20 minutes, but if you want a shorter run time, reduce the `experiment_timeout_hours` parameter.\n",
"\n",
"\n",
"|Property| Value in this tutorial |Description|\n",
"|----|----|---|\n",
"|**iteration_timeout_minutes**|10|Time limit in minutes for each iteration. Increase this value for larger datasets that need more time for each iteration.|\n",
"|**experiment_timeout_hours**|0.3|Maximum amount of time in hours that all iterations combined can take before the experiment terminates.|\n",
"|**enable_early_stopping**|True|Flag to enable early termination if the score is not improving in the short term.|\n",
"|**primary_metric**| spearman_correlation | Metric that you want to optimize. The best-fit model will be chosen based on this metric.|\n",
"|**featurization**| auto | By using auto, the experiment can preprocess the input data (handling missing data, converting text to numeric, etc.)|\n",
"|**verbosity**| logging.INFO | Controls the level of logging.|\n",
"|**n_cross_validations**|5|Number of cross-validation splits to perform when validation data is not specified.|"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import logging\n",
"\n",
"automl_settings = {\n",
" \"iteration_timeout_minutes\": 10,\n",
" \"experiment_timeout_hours\": 0.3,\n",
" \"enable_early_stopping\": True,\n",
" \"primary_metric\": 'spearman_correlation',\n",
" \"featurization\": 'auto',\n",
" \"verbosity\": logging.INFO,\n",
" \"n_cross_validations\": 5\n",
"}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use your defined training settings as a `**kwargs` parameter to an `AutoMLConfig` object. Additionally, specify your training data and the type of model, which is `regression` in this case."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.train.automl import AutoMLConfig\n",
"\n",
"automl_config = AutoMLConfig(task='regression',\n",
" debug_log='automated_ml_errors.log',\n",
" training_data=x_train,\n",
" label_column_name=\"totalAmount\",\n",
" **automl_settings)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Automated machine learning pre-processing steps (feature normalization, handling missing data, converting text to numeric, etc.) become part of the underlying model. When using the model for predictions, the same pre-processing steps applied during training are applied to your input data automatically."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Train the automatic regression model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create an experiment object in your workspace. An experiment acts as a container for your individual runs. Pass the defined `automl_config` object to the experiment, and set the output to `True` to view progress during the run. \n",
"\n",
"After starting the experiment, the output shown updates live as the experiment runs. For each iteration, you see the model type, the run duration, and the training accuracy. The field `BEST` tracks the best running training score based on your metric type."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.core.experiment import Experiment\n",
"experiment = Experiment(ws, \"Tutorial-NYCTaxi\")\n",
"local_run = experiment.submit(automl_config, show_output=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Explore the results"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Explore the results of automatic training with a [Jupyter widget](https://docs.microsoft.com/python/api/azureml-widgets/azureml.widgets?view=azure-ml-py). The widget allows you to see a graph and table of all individual run iterations, along with training accuracy metrics and metadata. Additionally, you can filter on different accuracy metrics than your primary metric with the dropdown selector."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from azureml.widgets import RunDetails\n",
"RunDetails(local_run).show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Retrieve the best model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Select the best model from your iterations. The `get_output` function returns the best run and the fitted model for the last fit invocation. By using the overloads on `get_output`, you can retrieve the best run and fitted model for any logged metric or a particular iteration."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"best_run, fitted_model = local_run.get_output()\n",
"print(best_run)\n",
"print(fitted_model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Test the best model accuracy"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the best model to run predictions on the test data set to predict taxi fares. The function `predict` uses the best model and predicts the values of y, **trip cost**, from the `x_test` data set. Print the first 10 predicted cost values from `y_predict`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"y_test = x_test.pop(\"totalAmount\")\n",
"\n",
"y_predict = fitted_model.predict(x_test)\n",
"print(y_predict[:10])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Calculate the `root mean squared error` of the results. Convert the `y_test` dataframe to a list to compare to the predicted values. The function `mean_squared_error` takes two arrays of values and calculates the average squared error between them. Taking the square root of the result gives an error in the same units as the y variable, **cost**. It indicates roughly how far the taxi fare predictions are from the actual fares."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.metrics import mean_squared_error\n",
"from math import sqrt\n",
"\n",
"y_actual = y_test.values.flatten().tolist()\n",
"rmse = sqrt(mean_squared_error(y_actual, y_predict))\n",
"rmse"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the following code to calculate mean absolute percent error (MAPE) by using the full `y_actual` and `y_predict` data sets. This metric calculates an absolute difference between each predicted and actual value and sums all the differences. Then it expresses that sum as a percent of the total of the actual values."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sum_actuals = sum_errors = 0\n",
"\n",
"for actual_val, predict_val in zip(y_actual, y_predict):\n",
" abs_error = actual_val - predict_val\n",
" if abs_error < 0:\n",
" abs_error = abs_error * -1\n",
"\n",
" sum_errors = sum_errors + abs_error\n",
" sum_actuals = sum_actuals + actual_val\n",
"\n",
"mean_abs_percent_error = sum_errors / sum_actuals\n",
"print(\"Model MAPE:\")\n",
"print(mean_abs_percent_error)\n",
"print()\n",
"print(\"Model Accuracy:\")\n",
"print(1 - mean_abs_percent_error)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"From the two prediction accuracy metrics, you see that the model is fairly good at predicting taxi fares from the data set's features, typically within +- $4.00, and approximately 15% error. \n",
"\n",
"The traditional machine learning model development process is highly resource-intensive, and requires significant domain knowledge and time investment to run and compare the results of dozens of models. Using automated machine learning is a great way to rapidly test many different models for your scenario."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up resources"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Do not complete this section if you plan on running other Azure Machine Learning service tutorials."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Stop the notebook VM"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you used a cloud notebook server, stop the VM when you are not using it to reduce cost."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. In your workspace, select **Compute**.\n",
"1. Select the **Notebook VMs** tab in the compute page.\n",
"1. From the list, select the VM.\n",
"1. Select **Stop**.\n",
"1. When you're ready to use the server again, select **Start**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Delete everything"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you don't plan to use the resources you created, delete them, so you don't incur any charges."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. In the Azure portal, select **Resource groups** on the far left.\n",
"1. From the list, select the resource group you created.\n",
"1. Select **Delete resource group**.\n",
"1. Enter the resource group name. Then select **Delete**.\n",
"\n",
"You can also keep the resource group but delete a single workspace. Display the workspace properties and select **Delete**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Next steps"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this automated machine learning tutorial, you did the following tasks:\n",
"\n",
"> * Configured a workspace and prepared data for an experiment.\n",
"> * Trained by using an automated regression model locally with custom parameters.\n",
"> * Explored and reviewed training results.\n",
"\n",
"[Deploy your model](https://docs.microsoft.com/azure/machine-learning/service/tutorial-deploy-models-with-aml) with Azure Machine Learning service."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"authors": [
{
"name": "jeffshep"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
},
"msauthor": "trbye",
"network_required": false
},
"nbformat": 4,
"nbformat_minor": 2
}

4
tutorials/regression-automl-nyc-taxi-data/regression-automated-ml.yml Normal file
View File

@@ -0,0 +1,4 @@
name: regression-automated-ml
dependencies:
- pip:
- azureml-sdk