mirror of
https://github.com/Azure/MachineLearningNotebooks.git
synced 2025-12-21 10:05:09 -05:00
Compare commits
29 Commits
azureml-sd
...
azureml-sd
| Author | SHA1 | Date | |
|---|---|---|---|
|
3531fe8a21 | ||
|
|
db6ae67940 | ||
|
|
2a479bb01e | ||
|
|
d05eec92af | ||
|
|
70fdab0a28 | ||
|
|
7ce5a43b58 | ||
|
|
d2a9dbb582 | ||
|
|
a5d774683d | ||
|
|
0e850f0917 | ||
|
|
59f34b7179 | ||
|
|
2a3cb69004 | ||
|
|
42894ff81a | ||
|
|
2163cab50b | ||
|
|
255edb04c0 | ||
|
|
cfce079278 | ||
|
|
ae6f067c81 | ||
|
|
1b7ff724f3 | ||
|
|
8bba850db1 | ||
|
|
b9e35ea0cb | ||
|
|
ffa28aa89c | ||
|
|
6ab85a20e3 | ||
|
|
486c44d157 | ||
|
|
cd80040dd8 | ||
|
|
465a5b13b1 | ||
|
|
dcd2d58880 | ||
|
|
93bf4393f2 | ||
|
|
d6ebb484a6 | ||
|
|
35afd43193 | ||
|
|
22597ac684 |
@@ -1,3 +1,4 @@
|
||||
|
||||
This software is made available to you on the condition that you agree to
|
||||
[your agreement][1] governing your use of Azure.
|
||||
If you do not have an existing agreement governing your use of Azure, you agree that
|
||||
20
README.md
20
README.md
@@ -54,5 +54,25 @@ Visit following repos to see projects contributed by Azure ML users:
|
||||
|
||||
- [Fine tune natural language processing models using Azure Machine Learning service](https://github.com/Microsoft/AzureML-BERT)
|
||||
- [Fashion MNIST with Azure ML SDK](https://github.com/amynic/azureml-sdk-fashion)
|
||||
|
||||
## Data/Telemetry
|
||||
This repository collects usage data and sends it to Mircosoft to help improve our products and services. Read Microsoft's [privacy statement to learn more](https://privacy.microsoft.com/en-US/privacystatement)
|
||||
|
||||
To opt out of tracking, please go to the raw markdown or .ipynb files and remove the following line of code:
|
||||
|
||||
```sh
|
||||
""
|
||||
```
|
||||
This URL will be slightly different depending on the file.
|
||||
|
||||
## Data/Telemetry
|
||||
This repository collects usage data and sends it to Mircosoft to help improve our products and services. Read Microsoft's [privacy statement to learn more](https://privacy.microsoft.com/en-US/privacystatement)
|
||||
|
||||
To opt out of tracking, please go to the raw markdown or .ipynb files and remove the following line of code:
|
||||
|
||||
```sh
|
||||
""
|
||||
```
|
||||
This URL will be slightly different depending on the file.
|
||||
|
||||
|
||||
|
||||
@@ -1,284 +1,293 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
||||
"\n",
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Configuration\n",
|
||||
"\n",
|
||||
"_**Setting up your Azure Machine Learning services workspace and configuring your notebook library**_\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Table of Contents\n",
|
||||
"\n",
|
||||
"1. [Introduction](#Introduction)\n",
|
||||
" 1. What is an Azure Machine Learning workspace\n",
|
||||
"1. [Setup](#Setup)\n",
|
||||
" 1. Azure subscription\n",
|
||||
" 1. Azure ML SDK and other library installation\n",
|
||||
" 1. Azure Container Instance registration\n",
|
||||
"1. [Configure your Azure ML Workspace](#Configure%20your%20Azure%20ML%20workspace)\n",
|
||||
" 1. Workspace parameters\n",
|
||||
" 1. Access your workspace\n",
|
||||
" 1. Create a new workspace\n",
|
||||
"1. [Next steps](#Next%20steps)\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Introduction\n",
|
||||
"\n",
|
||||
"This notebook configures your library of notebooks to connect to an Azure Machine Learning (ML) workspace. In this case, a library contains all of the notebooks in the current folder and any nested folders. You can configure this notebook library to use an existing workspace or create a new workspace.\n",
|
||||
"\n",
|
||||
"Typically you will need to run this notebook only once per notebook library as all other notebooks will use connection information that is written here. If you want to redirect your notebook library to work with a different workspace, then you should re-run this notebook.\n",
|
||||
"\n",
|
||||
"In this notebook you will\n",
|
||||
"* Learn about getting an Azure subscription\n",
|
||||
"* Specify your workspace parameters\n",
|
||||
"* Access or create your workspace\n",
|
||||
"* Add a default compute cluster for your workspace\n",
|
||||
"\n",
|
||||
"### What is an Azure Machine Learning workspace\n",
|
||||
"\n",
|
||||
"An Azure ML Workspace is an Azure resource that organizes and coordinates the actions of many other Azure resources to assist in executing and sharing machine learning workflows. In particular, an Azure ML Workspace coordinates storage, databases, and compute resources providing added functionality for machine learning experimentation, deployment, inferencing, and the monitoring of deployed models."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"This section describes activities required before you can access any Azure ML services functionality."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 1. Azure Subscription\n",
|
||||
"\n",
|
||||
"In order to create an Azure ML Workspace, first you need access to an Azure subscription. An Azure subscription allows you to manage storage, compute, and other assets in the Azure cloud. You can [create a new subscription](https://azure.microsoft.com/en-us/free/) or access existing subscription information from the [Azure portal](https://portal.azure.com). Later in this notebook you will need information such as your subscription ID in order to create and access AML workspaces.\n",
|
||||
"\n",
|
||||
"### 2. Azure ML SDK and other library installation\n",
|
||||
"\n",
|
||||
"If you are running in your own environment, follow [SDK installation instructions](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-environment). If you are running in Azure Notebooks or another Microsoft managed environment, the SDK is already installed.\n",
|
||||
"\n",
|
||||
"Also install following libraries to your environment. Many of the example notebooks depend on them\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"(myenv) $ conda install -y matplotlib tqdm scikit-learn\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Once installation is complete, the following cell checks the Azure ML SDK version:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"install"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import azureml.core\n",
|
||||
"\n",
|
||||
"print(\"This notebook was created using version 1.0.39 of the Azure ML SDK\")\n",
|
||||
"print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you are using an older version of the SDK then this notebook was created using, you should upgrade your SDK.\n",
|
||||
"\n",
|
||||
"### 3. Azure Container Instance registration\n",
|
||||
"Azure Machine Learning uses of [Azure Container Instance (ACI)](https://azure.microsoft.com/services/container-instances) to deploy dev/test web services. An Azure subscription needs to be registered to use ACI. If you or the subscription owner have not yet registered ACI on your subscription, you will need to use the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) and execute the following commands. Note that if you ran through the AML [quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-get-started) you have already registered ACI. \n",
|
||||
"\n",
|
||||
"```shell\n",
|
||||
"# check to see if ACI is already registered\n",
|
||||
"(myenv) $ az provider show -n Microsoft.ContainerInstance -o table\n",
|
||||
"\n",
|
||||
"# if ACI is not registered, run this command.\n",
|
||||
"# note you need to be the subscription owner in order to execute this command successfully.\n",
|
||||
"(myenv) $ az provider register -n Microsoft.ContainerInstance\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"---"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Configure your Azure ML workspace\n",
|
||||
"\n",
|
||||
"### Workspace parameters\n",
|
||||
"\n",
|
||||
"To use an AML Workspace, you will need to import the Azure ML SDK and supply the following information:\n",
|
||||
"* Your subscription id\n",
|
||||
"* A resource group name\n",
|
||||
"* (optional) The region that will host your workspace\n",
|
||||
"* A name for your workspace\n",
|
||||
"\n",
|
||||
"You can get your subscription ID from the [Azure portal](https://portal.azure.com).\n",
|
||||
"\n",
|
||||
"You will also need access to a [_resource group_](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview#resource-groups), which organizes Azure resources and provides a default region for the resources in a group. You can see what resource groups to which you have access, or create a new one in the [Azure portal](https://portal.azure.com). If you don't have a resource group, the create workspace command will create one for you using the name you provide.\n",
|
||||
"\n",
|
||||
"The region to host your workspace will be used if you are creating a new workspace. You do not need to specify this if you are using an existing workspace. You can find the list of supported regions [here](https://azure.microsoft.com/en-us/global-infrastructure/services/?products=machine-learning-service). You should pick a region that is close to your location or that contains your data.\n",
|
||||
"\n",
|
||||
"The name for your workspace is unique within the subscription and should be descriptive enough to discern among other AML Workspaces. The subscription may be used only by you, or it may be used by your department or your entire enterprise, so choose a name that makes sense for your situation.\n",
|
||||
"\n",
|
||||
"The following cell allows you to specify your workspace parameters. This cell uses the python method `os.getenv` to read values from environment variables which is useful for automation. If no environment variable exists, the parameters will be set to the specified default values. \n",
|
||||
"\n",
|
||||
"If you ran the Azure Machine Learning [quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-get-started) in Azure Notebooks, you already have a configured workspace! You can go to your Azure Machine Learning Getting Started library, view *config.json* file, and copy-paste the values for subscription ID, resource group and workspace name below.\n",
|
||||
"\n",
|
||||
"Replace the default values in the cell below with your workspace parameters"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"subscription_id = os.getenv(\"SUBSCRIPTION_ID\", default=\"<my-subscription-id>\")\n",
|
||||
"resource_group = os.getenv(\"RESOURCE_GROUP\", default=\"<my-resource-group>\")\n",
|
||||
"workspace_name = os.getenv(\"WORKSPACE_NAME\", default=\"<my-workspace-name>\")\n",
|
||||
"workspace_region = os.getenv(\"WORKSPACE_REGION\", default=\"eastus2\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Access your workspace\n",
|
||||
"\n",
|
||||
"The following cell uses the Azure ML SDK to attempt to load the workspace specified by your parameters. If this cell succeeds, your notebook library will be configured to access the workspace from all notebooks using the `Workspace.from_config()` method. The cell can fail if the specified workspace doesn't exist or you don't have permissions to access it. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name = workspace_name)\n",
|
||||
" # write the details of the workspace to a configuration file to the notebook library\n",
|
||||
" ws.write_config()\n",
|
||||
" print(\"Workspace configuration succeeded. Skip the workspace creation steps below\")\n",
|
||||
"except:\n",
|
||||
" print(\"Workspace not accessible. Change your parameters or create a new workspace below\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Create a new workspace\n",
|
||||
"\n",
|
||||
"If you don't have an existing workspace and are the owner of the subscription or resource group, you can create a new workspace. If you don't have a resource group, the create workspace command will create one for you using the name you provide.\n",
|
||||
"\n",
|
||||
"**Note**: As with other Azure services, there are limits on certain resources (for example AmlCompute quota) associated with the Azure ML service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota.\n",
|
||||
"\n",
|
||||
"This cell will create an Azure ML workspace for you in a subscription provided you have the correct permissions.\n",
|
||||
"\n",
|
||||
"This will fail if:\n",
|
||||
"* You do not have permission to create a workspace in the resource group\n",
|
||||
"* You do not have permission to create a resource group if it's non-existing.\n",
|
||||
"* You are not a subscription owner or contributor and no Azure ML workspaces have ever been created in this subscription\n",
|
||||
"\n",
|
||||
"If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to provision the required resources."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"create workspace"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"# Create the workspace using the specified parameters\n",
|
||||
"ws = Workspace.create(name = workspace_name,\n",
|
||||
" subscription_id = subscription_id,\n",
|
||||
" resource_group = resource_group, \n",
|
||||
" location = workspace_region,\n",
|
||||
" create_resource_group = True,\n",
|
||||
" exist_ok = True)\n",
|
||||
"ws.get_details()\n",
|
||||
"\n",
|
||||
"# write the details of the workspace to a configuration file to the notebook library\n",
|
||||
"ws.write_config()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Next steps\n",
|
||||
"\n",
|
||||
"In this notebook you configured this notebook library to connect easily to an Azure ML workspace. You can copy this notebook to your own libraries to connect them to you workspace, or use it to bootstrap new workspaces completely.\n",
|
||||
"\n",
|
||||
"If you came here from another notebook, you can return there and complete that exercise, or you can try out the [Tutorials](./tutorials) or jump into \"how-to\" notebooks and start creating and deploying models. A good place to start is the [train within notebook](./how-to-use-azureml/training/train-within-notebook) example that walks through a simplified but complete end to end machine learning process."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"authors": [
|
||||
{
|
||||
"name": "roastala"
|
||||
}
|
||||
],
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.6",
|
||||
"language": "python",
|
||||
"name": "python36"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.5"
|
||||
}
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
||||
"\n",
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Configuration\n",
|
||||
"\n",
|
||||
"_**Setting up your Azure Machine Learning services workspace and configuring your notebook library**_\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Table of Contents\n",
|
||||
"\n",
|
||||
"1. [Introduction](#Introduction)\n",
|
||||
" 1. What is an Azure Machine Learning workspace\n",
|
||||
"1. [Setup](#Setup)\n",
|
||||
" 1. Azure subscription\n",
|
||||
" 1. Azure ML SDK and other library installation\n",
|
||||
" 1. Azure Container Instance registration\n",
|
||||
"1. [Configure your Azure ML Workspace](#Configure%20your%20Azure%20ML%20workspace)\n",
|
||||
" 1. Workspace parameters\n",
|
||||
" 1. Access your workspace\n",
|
||||
" 1. Create a new workspace\n",
|
||||
"1. [Next steps](#Next%20steps)\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Introduction\n",
|
||||
"\n",
|
||||
"This notebook configures your library of notebooks to connect to an Azure Machine Learning (ML) workspace. In this case, a library contains all of the notebooks in the current folder and any nested folders. You can configure this notebook library to use an existing workspace or create a new workspace.\n",
|
||||
"\n",
|
||||
"Typically you will need to run this notebook only once per notebook library as all other notebooks will use connection information that is written here. If you want to redirect your notebook library to work with a different workspace, then you should re-run this notebook.\n",
|
||||
"\n",
|
||||
"In this notebook you will\n",
|
||||
"* Learn about getting an Azure subscription\n",
|
||||
"* Specify your workspace parameters\n",
|
||||
"* Access or create your workspace\n",
|
||||
"* Add a default compute cluster for your workspace\n",
|
||||
"\n",
|
||||
"### What is an Azure Machine Learning workspace\n",
|
||||
"\n",
|
||||
"An Azure ML Workspace is an Azure resource that organizes and coordinates the actions of many other Azure resources to assist in executing and sharing machine learning workflows. In particular, an Azure ML Workspace coordinates storage, databases, and compute resources providing added functionality for machine learning experimentation, deployment, inferencing, and the monitoring of deployed models."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup\n",
|
||||
"\n",
|
||||
"This section describes activities required before you can access any Azure ML services functionality."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### 1. Azure Subscription\n",
|
||||
"\n",
|
||||
"In order to create an Azure ML Workspace, first you need access to an Azure subscription. An Azure subscription allows you to manage storage, compute, and other assets in the Azure cloud. You can [create a new subscription](https://azure.microsoft.com/en-us/free/) or access existing subscription information from the [Azure portal](https://portal.azure.com). Later in this notebook you will need information such as your subscription ID in order to create and access AML workspaces.\n",
|
||||
"\n",
|
||||
"### 2. Azure ML SDK and other library installation\n",
|
||||
"\n",
|
||||
"If you are running in your own environment, follow [SDK installation instructions](https://docs.microsoft.com/azure/machine-learning/service/how-to-configure-environment). If you are running in Azure Notebooks or another Microsoft managed environment, the SDK is already installed.\n",
|
||||
"\n",
|
||||
"Also install following libraries to your environment. Many of the example notebooks depend on them\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"(myenv) $ conda install -y matplotlib tqdm scikit-learn\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Once installation is complete, the following cell checks the Azure ML SDK version:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"install"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import azureml.core\n",
|
||||
"\n",
|
||||
"print(\"This notebook was created using version 1.0.41 of the Azure ML SDK\")\n",
|
||||
"print(\"You are currently using version\", azureml.core.VERSION, \"of the Azure ML SDK\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If you are using an older version of the SDK then this notebook was created using, you should upgrade your SDK.\n",
|
||||
"\n",
|
||||
"### 3. Azure Container Instance registration\n",
|
||||
"Azure Machine Learning uses of [Azure Container Instance (ACI)](https://azure.microsoft.com/services/container-instances) to deploy dev/test web services. An Azure subscription needs to be registered to use ACI. If you or the subscription owner have not yet registered ACI on your subscription, you will need to use the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) and execute the following commands. Note that if you ran through the AML [quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-get-started) you have already registered ACI. \n",
|
||||
"\n",
|
||||
"```shell\n",
|
||||
"# check to see if ACI is already registered\n",
|
||||
"(myenv) $ az provider show -n Microsoft.ContainerInstance -o table\n",
|
||||
"\n",
|
||||
"# if ACI is not registered, run this command.\n",
|
||||
"# note you need to be the subscription owner in order to execute this command successfully.\n",
|
||||
"(myenv) $ az provider register -n Microsoft.ContainerInstance\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"---"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Configure your Azure ML workspace\n",
|
||||
"\n",
|
||||
"### Workspace parameters\n",
|
||||
"\n",
|
||||
"To use an AML Workspace, you will need to import the Azure ML SDK and supply the following information:\n",
|
||||
"* Your subscription id\n",
|
||||
"* A resource group name\n",
|
||||
"* (optional) The region that will host your workspace\n",
|
||||
"* A name for your workspace\n",
|
||||
"\n",
|
||||
"You can get your subscription ID from the [Azure portal](https://portal.azure.com).\n",
|
||||
"\n",
|
||||
"You will also need access to a [_resource group_](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview#resource-groups), which organizes Azure resources and provides a default region for the resources in a group. You can see what resource groups to which you have access, or create a new one in the [Azure portal](https://portal.azure.com). If you don't have a resource group, the create workspace command will create one for you using the name you provide.\n",
|
||||
"\n",
|
||||
"The region to host your workspace will be used if you are creating a new workspace. You do not need to specify this if you are using an existing workspace. You can find the list of supported regions [here](https://azure.microsoft.com/en-us/global-infrastructure/services/?products=machine-learning-service). You should pick a region that is close to your location or that contains your data.\n",
|
||||
"\n",
|
||||
"The name for your workspace is unique within the subscription and should be descriptive enough to discern among other AML Workspaces. The subscription may be used only by you, or it may be used by your department or your entire enterprise, so choose a name that makes sense for your situation.\n",
|
||||
"\n",
|
||||
"The following cell allows you to specify your workspace parameters. This cell uses the python method `os.getenv` to read values from environment variables which is useful for automation. If no environment variable exists, the parameters will be set to the specified default values. \n",
|
||||
"\n",
|
||||
"If you ran the Azure Machine Learning [quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-get-started) in Azure Notebooks, you already have a configured workspace! You can go to your Azure Machine Learning Getting Started library, view *config.json* file, and copy-paste the values for subscription ID, resource group and workspace name below.\n",
|
||||
"\n",
|
||||
"Replace the default values in the cell below with your workspace parameters"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"subscription_id = os.getenv(\"SUBSCRIPTION_ID\", default=\"<my-subscription-id>\")\n",
|
||||
"resource_group = os.getenv(\"RESOURCE_GROUP\", default=\"<my-resource-group>\")\n",
|
||||
"workspace_name = os.getenv(\"WORKSPACE_NAME\", default=\"<my-workspace-name>\")\n",
|
||||
"workspace_region = os.getenv(\"WORKSPACE_REGION\", default=\"eastus2\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Access your workspace\n",
|
||||
"\n",
|
||||
"The following cell uses the Azure ML SDK to attempt to load the workspace specified by your parameters. If this cell succeeds, your notebook library will be configured to access the workspace from all notebooks using the `Workspace.from_config()` method. The cell can fail if the specified workspace doesn't exist or you don't have permissions to access it. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name = workspace_name)\n",
|
||||
" # write the details of the workspace to a configuration file to the notebook library\n",
|
||||
" ws.write_config()\n",
|
||||
" print(\"Workspace configuration succeeded. Skip the workspace creation steps below\")\n",
|
||||
"except:\n",
|
||||
" print(\"Workspace not accessible. Change your parameters or create a new workspace below\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Create a new workspace\n",
|
||||
"\n",
|
||||
"If you don't have an existing workspace and are the owner of the subscription or resource group, you can create a new workspace. If you don't have a resource group, the create workspace command will create one for you using the name you provide.\n",
|
||||
"\n",
|
||||
"**Note**: The Workspace creation command will create default CPU and GPU compute clusters for you. As with other Azure services, there are limits on certain resources (for example AmlCompute quota) associated with the Azure ML service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota.\n",
|
||||
"\n",
|
||||
"This cell will create an Azure ML workspace for you in a subscription provided you have the correct permissions.\n",
|
||||
"\n",
|
||||
"This will fail if:\n",
|
||||
"* You do not have permission to create a workspace in the resource group\n",
|
||||
"* You do not have permission to create a resource group if it's non-existing.\n",
|
||||
"* You are not a subscription owner or contributor and no Azure ML workspaces have ever been created in this subscription\n",
|
||||
"\n",
|
||||
"If workspace creation fails, please work with your IT admin to provide you with the appropriate permissions or to provision the required resources."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"create workspace"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"# Create the workspace using the specified parameters\n",
|
||||
"ws = Workspace.create(name = workspace_name,\n",
|
||||
" subscription_id = subscription_id,\n",
|
||||
" resource_group = resource_group, \n",
|
||||
" location = workspace_region,\n",
|
||||
" default_cpu_compute_target=Workspace.DEFAULT_CPU_CLUSTER_CONFIGURATION,\n",
|
||||
" default_gpu_compute_target=Workspace.DEFAULT_GPU_CLUSTER_CONFIGURATION,\n",
|
||||
" create_resource_group = True,\n",
|
||||
" exist_ok = True)\n",
|
||||
"ws.get_details()\n",
|
||||
"\n",
|
||||
"# write the details of the workspace to a configuration file to the notebook library\n",
|
||||
"ws.write_config()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Next steps\n",
|
||||
"\n",
|
||||
"In this notebook you configured this notebook library to connect easily to an Azure ML workspace. You can copy this notebook to your own libraries to connect them to you workspace, or use it to bootstrap new workspaces completely.\n",
|
||||
"\n",
|
||||
"If you came here from another notebook, you can return there and complete that exercise, or you can try out the [Tutorials](./tutorials) or jump into \"how-to\" notebooks and start creating and deploying models. A good place to start is the [train within notebook](./how-to-use-azureml/training/train-within-notebook) example that walks through a simplified but complete end to end machine learning process."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"authors": [
|
||||
{
|
||||
"name": "roastala"
|
||||
}
|
||||
],
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.6",
|
||||
"language": "python",
|
||||
"name": "python36"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Learn how to use Azure Machine Learning services for experimentation and model management.
|
||||
|
||||
If you are using an Azure Machine Learning Notebook VM, you are all set. Otherwise, go through the [configuration Notebook](../configuration.ipynb) first if you haven't already to establish your connection to the AzureML Workspace. Then, run the notebooks in following recommended order.
|
||||
As a pre-requisite, run the [configuration Notebook](../configuration.ipynb) notebook first to set up your Azure ML Workspace. Then, run the notebooks in following recommended order.
|
||||
|
||||
* [train-within-notebook](./training/train-within-notebook): Train a model hile tracking run history, and learn how to deploy the model as web service to Azure Container Instance.
|
||||
* [train-on-local](./training/train-on-local): Learn how to submit a run to local computer and use Azure ML managed run configuration.
|
||||
@@ -15,6 +15,3 @@ If you are using an Azure Machine Learning Notebook VM, you are all set. Otherw
|
||||
* [enable-app-insights-in-production-service](./deployment/enable-app-insights-in-production-service) Learn how to use App Insights with production web service.
|
||||
|
||||
Find quickstarts, end-to-end tutorials, and how-tos on the [official documentation site for Azure Machine Learning service](https://docs.microsoft.com/en-us/azure/machine-learning/service/).
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ dependencies:
|
||||
- urllib3<1.24
|
||||
- scipy>=1.0.0,<=1.1.0
|
||||
- scikit-learn>=0.19.0,<=0.20.3
|
||||
- pandas>=0.22.0,<0.23.0
|
||||
- pandas>=0.22.0,<=0.23.4
|
||||
- py-xgboost<=0.80
|
||||
|
||||
- pip:
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
name: azure_automl
|
||||
dependencies:
|
||||
# The python interpreter version.
|
||||
# Currently Azure ML only supports 3.5.2 and later.
|
||||
- nomkl
|
||||
- python>=3.5.2,<3.6.8
|
||||
- nb_conda
|
||||
- matplotlib==2.1.0
|
||||
- numpy>=1.11.0,<=1.16.2
|
||||
- cython
|
||||
- urllib3<1.24
|
||||
- scipy>=1.0.0,<=1.1.0
|
||||
- scikit-learn>=0.19.0,<=0.20.3
|
||||
- pandas>=0.22.0,<0.23.0
|
||||
- py-xgboost<=0.80
|
||||
|
||||
- pip:
|
||||
# Required packages for AzureML execution, history, and data preparation.
|
||||
- azureml-sdk[automl,explain]
|
||||
- azureml-widgets
|
||||
- pandas_ml
|
||||
|
||||
@@ -12,7 +12,7 @@ fi
|
||||
|
||||
if [ "$AUTOML_ENV_FILE" == "" ]
|
||||
then
|
||||
AUTOML_ENV_FILE="automl_env.yml"
|
||||
AUTOML_ENV_FILE="automl_env_mac.yml"
|
||||
fi
|
||||
|
||||
if [ ! -f $AUTOML_ENV_FILE ]; then
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -66,11 +73,12 @@
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"from sklearn import datasets\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"\n",
|
||||
"import azureml.core\n",
|
||||
"from azureml.core.experiment import Experiment\n",
|
||||
"from azureml.core.workspace import Workspace\n",
|
||||
"from azureml.train.automl import AutoMLConfig"
|
||||
"from azureml.train.automl import AutoMLConfig, constants"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -106,7 +114,7 @@
|
||||
"source": [
|
||||
"## Data\n",
|
||||
"\n",
|
||||
"This uses scikit-learn's [load_digits](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html) method."
|
||||
"This uses scikit-learn's [load_iris](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html) method."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -115,11 +123,17 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"digits = datasets.load_digits()\n",
|
||||
"iris = datasets.load_iris()\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(iris.data, \n",
|
||||
" iris.target, \n",
|
||||
" test_size=0.2, \n",
|
||||
" random_state=0)\n",
|
||||
"\n",
|
||||
"# Exclude the first 100 rows from training so that they can be used for test.\n",
|
||||
"X_train = digits.data[100:,:]\n",
|
||||
"y_train = digits.target[100:]"
|
||||
"# Convert the X_train and X_test to pandas DataFrame and set column names,\n",
|
||||
"# This is needed for initializing the input variable names of ONNX model, \n",
|
||||
"# and the prediction with the ONNX model using the inference helper.\n",
|
||||
"X_train = pd.DataFrame(X_train, columns=['c1', 'c2', 'c3', 'c4'])\n",
|
||||
"X_test = pd.DataFrame(X_test, columns=['c1', 'c2', 'c3', 'c4'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -155,9 +169,10 @@
|
||||
" primary_metric = 'AUC_weighted',\n",
|
||||
" iteration_timeout_minutes = 60,\n",
|
||||
" iterations = 10,\n",
|
||||
" verbosity = logging.INFO,\n",
|
||||
" verbosity = logging.INFO, \n",
|
||||
" X = X_train, \n",
|
||||
" y = y_train,\n",
|
||||
" preprocess=True,\n",
|
||||
" enable_onnx_compatible_models=True,\n",
|
||||
" path = project_folder)"
|
||||
]
|
||||
@@ -253,6 +268,65 @@
|
||||
"onnx_fl_path = \"./best_model.onnx\"\n",
|
||||
"OnnxConverter.save_onnx_model(onnx_mdl, onnx_fl_path)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Predict with the ONNX model, using onnxruntime package"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import sys\n",
|
||||
"import json\n",
|
||||
"from azureml.automl.core.onnx_convert import OnnxConvertConstants\n",
|
||||
"\n",
|
||||
"if sys.version_info < OnnxConvertConstants.OnnxIncompatiblePythonVersion:\n",
|
||||
" python_version_compatible = True\n",
|
||||
"else:\n",
|
||||
" python_version_compatible = False\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" import onnxruntime\n",
|
||||
" from azureml.automl.core.onnx_convert import OnnxInferenceHelper \n",
|
||||
" onnxrt_present = True\n",
|
||||
"except ImportError:\n",
|
||||
" onnxrt_present = False\n",
|
||||
"\n",
|
||||
"def get_onnx_res(run):\n",
|
||||
" res_path = '_debug_y_trans_converter.json'\n",
|
||||
" run.download_file(name=constants.MODEL_RESOURCE_PATH_ONNX, output_file_path=res_path)\n",
|
||||
" with open(res_path) as f:\n",
|
||||
" onnx_res = json.load(f)\n",
|
||||
" return onnx_res\n",
|
||||
"\n",
|
||||
"if onnxrt_present and python_version_compatible: \n",
|
||||
" mdl_bytes = onnx_mdl.SerializeToString()\n",
|
||||
" onnx_res = get_onnx_res(best_run)\n",
|
||||
"\n",
|
||||
" onnxrt_helper = OnnxInferenceHelper(mdl_bytes, onnx_res)\n",
|
||||
" pred_onnx, pred_prob_onnx = onnxrt_helper.predict(X_test)\n",
|
||||
"\n",
|
||||
" print(pred_onnx)\n",
|
||||
" print(pred_prob_onnx)\n",
|
||||
"else:\n",
|
||||
" if not python_version_compatible:\n",
|
||||
" print('Please use Python version 3.6 to run the inference helper.') \n",
|
||||
" if not onnxrt_present:\n",
|
||||
" print('Please install the onnxruntime package to do the prediction with ONNX model.')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -112,7 +119,7 @@
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Create or Attach existing AmlCompute\n",
|
||||
"You will need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture#compute-target) for your AutoML run. In this tutorial, you attach the default `AmlCompute` as your training compute resource.\n",
|
||||
"You will need to create a [compute target](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture#compute-target) for your AutoML run. In this tutorial, you create an AmlCompute as your training compute resource.\n",
|
||||
"\n",
|
||||
"As with other Azure services, there are limits on certain resources (e.g. AmlCompute) associated with the Azure Machine Learning service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota."
|
||||
]
|
||||
@@ -123,16 +130,38 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"compute_target = ws.get_default_compute_target(\"CPU\")"
|
||||
"from azureml.core.compute import AmlCompute\n",
|
||||
"from azureml.core.compute import ComputeTarget\n",
|
||||
"\n",
|
||||
"# Choose a name for your cluster.\n",
|
||||
"amlcompute_cluster_name = \"cpucluster\"\n",
|
||||
"\n",
|
||||
"found = False\n",
|
||||
"\n",
|
||||
"# Check if this compute target already exists in the workspace.\n",
|
||||
"\n",
|
||||
"cts = ws.compute_targets\n",
|
||||
"if amlcompute_cluster_name in cts and cts[amlcompute_cluster_name].type == 'AmlCompute':\n",
|
||||
" found = True\n",
|
||||
" print('Found existing compute target.')\n",
|
||||
" compute_target = cts[amlcompute_cluster_name]\n",
|
||||
"\n",
|
||||
"if not found:\n",
|
||||
" print('Creating a new compute target...')\n",
|
||||
" provisioning_config = AmlCompute.provisioning_configuration(vm_size = \"STANDARD_D2_V2\", # for GPU, use \"STANDARD_NC6\"\n",
|
||||
" #vm_priority = 'lowpriority', # optional\n",
|
||||
" max_nodes = 6)\n",
|
||||
"\n",
|
||||
" # Create the cluster.\\n\",\n",
|
||||
" compute_target = ComputeTarget.create(ws, amlcompute_cluster_name, provisioning_config)\n",
|
||||
"\n",
|
||||
" # Can poll for a minimum number of nodes and for a specific timeout.\n",
|
||||
" # If no min_node_count is provided, it will use the scale settings for the cluster.\n",
|
||||
" compute_target.wait_for_completion(show_output = True, min_node_count = None, timeout_in_minutes = 20)\n",
|
||||
"\n",
|
||||
" # For a more detailed view of current AmlCompute status, use get_status()."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -26,4 +26,8 @@ You can use Azure Databricks as a compute target from [Azure Machine Learning Pi
|
||||
|
||||
For more on SDK concepts, please refer to [notebooks](https://github.com/Azure/MachineLearningNotebooks).
|
||||
|
||||
**Please let us know your feedback.**
|
||||
**Please let us know your feedback.**
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -11,6 +11,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -333,6 +340,13 @@
|
||||
"source": [
|
||||
"dbutils.notebook.exit(\"success\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -11,6 +11,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -277,6 +284,13 @@
|
||||
"#comment to not delete the web service\n",
|
||||
"myservice.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -11,6 +11,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -203,6 +210,13 @@
|
||||
"#model.delete()\n",
|
||||
"aks_target.delete() "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -11,6 +11,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -139,6 +146,13 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -11,6 +11,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -143,6 +150,13 @@
|
||||
" 'Subscription id: ' + ws.subscription_id, \n",
|
||||
" 'Resource group: ' + ws.resource_group, sep = '\\n')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -660,6 +660,13 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -796,6 +796,13 @@
|
||||
"source": [
|
||||
"myservice.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -677,6 +677,13 @@
|
||||
"# Next: ADLA as a Compute Target\n",
|
||||
"To use ADLA as a compute target from Azure Machine Learning Pipeline, a AdlaStep is used. This [notebook](./aml-pipelines-use-adla-as-compute-target.ipynb) demonstrates the use of AdlaStep in Azure Machine Learning Pipeline."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -9,6 +9,13 @@
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
@@ -255,7 +255,7 @@
|
||||
"source": [
|
||||
"from azureml.core.conda_dependencies import CondaDependencies \n",
|
||||
"\n",
|
||||
"myenv = CondaDependencies.create(pip_packages=[\"numpy\",\"onnxruntime\",\"azureml-core\"])\n",
|
||||
"myenv = CondaDependencies.create(pip_packages=[\"numpy\",\"onnxruntime==0.4.0\",\"azureml-core\"])\n",
|
||||
"\n",
|
||||
"with open(\"myenv.yml\",\"w\") as f:\n",
|
||||
" f.write(myenv.serialize_to_string())"
|
||||
|
||||
@@ -63,7 +63,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"aml_compute = ws.get_default_compute_target(\"CPU\")"
|
||||
"aml_compute_target = ws.get_default_compute_target(\"CPU\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -284,7 +284,10 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Disable the schedule"
|
||||
"### Disable the schedule\n",
|
||||
"It is important to note the best practice of disabling schedules when not in use.\n",
|
||||
"The number of schedule triggers allowed per month per region per subscription is 100,000.\n",
|
||||
"This is calculated using the project trigger counts for all active schedules."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -8,5 +8,6 @@ Follow these sample notebooks to learn:
|
||||
4. [Train on AmlCompute](train-on-amlcompute): train a model using an AmlCompute cluster as compute target.
|
||||
5. [Train in an HDI Spark cluster](train-in-spark): train a Spark ML model using an HDInsight Spark cluster as compute target.
|
||||
6. [Logging API](logging-api): experiment with various logging functions to create runs and automatically generate graphs.
|
||||
7. [Train and hyperparameter tune on Iris Dataset with Scikit-learn](train-hyperparameter-tune-deploy-with-sklearn): train a model using the Scikit-learn estimator and tune hyperparameters with Hyperdrive.
|
||||
|
||||
|
||||
@@ -100,7 +100,7 @@
|
||||
"\n",
|
||||
"# Check core SDK version number\n",
|
||||
"\n",
|
||||
"print(\"This notebook was created using SDK version 1.0.39, you are currently running version\", azureml.core.VERSION)"
|
||||
"print(\"This notebook was created using SDK version 1.0.41, you are currently running version\", azureml.core.VERSION)"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -164,6 +164,30 @@
|
||||
"shutil.copy('train.py', project_folder)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Create environment\n",
|
||||
"\n",
|
||||
"Create Docker based environment with scikit-learn installed."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Environment\n",
|
||||
"from azureml.core.conda_dependencies import CondaDependencies\n",
|
||||
"\n",
|
||||
"myenv = Environment(\"myenv\")\n",
|
||||
"\n",
|
||||
"myenv.docker.enabled = True\n",
|
||||
"myenv.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -179,38 +203,32 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.runconfig import RunConfiguration\n",
|
||||
"from azureml.core.conda_dependencies import CondaDependencies\n",
|
||||
"from azureml.core import ScriptRunConfig\n",
|
||||
"from azureml.core.runconfig import DEFAULT_CPU_IMAGE\n",
|
||||
"\n",
|
||||
"# create a new runconfig object\n",
|
||||
"run_config = RunConfiguration()\n",
|
||||
"src = ScriptRunConfig(source_directory=project_folder, script='train.py')\n",
|
||||
"\n",
|
||||
"default_compute_target = ws.get_default_compute_target(type=\"CPU\")\n",
|
||||
"# Use default compute target\n",
|
||||
"src.run_config.target = ws.get_default_compute_target(type=\"CPU\").name\n",
|
||||
"\n",
|
||||
"# signal that you want to use AmlCompute to execute script.\n",
|
||||
"run_config.target = default_compute_target.name\n",
|
||||
"\n",
|
||||
"# enable Docker \n",
|
||||
"run_config.environment.docker.enabled = True\n",
|
||||
"\n",
|
||||
"# set Docker base image to the default CPU-based image\n",
|
||||
"run_config.environment.docker.base_image = DEFAULT_CPU_IMAGE\n",
|
||||
"\n",
|
||||
"# use conda_dependencies.yml to create a conda environment in the Docker image for execution\n",
|
||||
"run_config.environment.python.user_managed_dependencies = False\n",
|
||||
"\n",
|
||||
"# specify CondaDependencies obj\n",
|
||||
"run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])\n",
|
||||
"\n",
|
||||
"# Now submit a run on AmlCompute\n",
|
||||
"from azureml.core.script_run_config import ScriptRunConfig\n",
|
||||
"\n",
|
||||
"script_run_config = ScriptRunConfig(source_directory=project_folder,\n",
|
||||
" script='train.py',\n",
|
||||
" run_config=run_config)\n",
|
||||
"\n",
|
||||
"run = experiment.submit(script_run_config)\n",
|
||||
"# Set environment\n",
|
||||
"src.run_config.environment = myenv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Submit run"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"run = experiment.submit(src)\n",
|
||||
"\n",
|
||||
"# Show run details\n",
|
||||
"run"
|
||||
@@ -292,27 +310,9 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.runconfig import RunConfiguration\n",
|
||||
"from azureml.core.conda_dependencies import CondaDependencies\n",
|
||||
"\n",
|
||||
"# create a new RunConfig object\n",
|
||||
"run_config = RunConfiguration(framework=\"python\")\n",
|
||||
"\n",
|
||||
"# Set compute target to AmlCompute target created in previous step\n",
|
||||
"run_config.target = cpu_cluster.name\n",
|
||||
"\n",
|
||||
"# enable Docker \n",
|
||||
"run_config.environment.docker.enabled = True\n",
|
||||
"\n",
|
||||
"# specify CondaDependencies obj\n",
|
||||
"run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])\n",
|
||||
"\n",
|
||||
"from azureml.core import Run\n",
|
||||
"from azureml.core import ScriptRunConfig\n",
|
||||
"\n",
|
||||
"src = ScriptRunConfig(source_directory=project_folder, \n",
|
||||
" script='train.py', \n",
|
||||
" run_config=run_config) \n",
|
||||
"# Set compute target to the one created in previous step\n",
|
||||
"src.run_config.target = cpu_cluster.name\n",
|
||||
" \n",
|
||||
"run = experiment.submit(config=src)\n",
|
||||
"run"
|
||||
]
|
||||
@@ -397,27 +397,9 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.runconfig import RunConfiguration\n",
|
||||
"from azureml.core.conda_dependencies import CondaDependencies\n",
|
||||
"\n",
|
||||
"# create a new RunConfig object\n",
|
||||
"run_config = RunConfiguration(framework=\"python\")\n",
|
||||
"\n",
|
||||
"# Set compute target to AmlCompute target created in previous step\n",
|
||||
"run_config.target = cpu_cluster.name\n",
|
||||
"\n",
|
||||
"# enable Docker \n",
|
||||
"run_config.environment.docker.enabled = True\n",
|
||||
"\n",
|
||||
"# specify CondaDependencies obj\n",
|
||||
"run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])\n",
|
||||
"\n",
|
||||
"from azureml.core import Run\n",
|
||||
"from azureml.core import ScriptRunConfig\n",
|
||||
"\n",
|
||||
"src = ScriptRunConfig(source_directory=project_folder, \n",
|
||||
" script='train.py', \n",
|
||||
" run_config=run_config) \n",
|
||||
"# Set compute target to the one created in previous step\n",
|
||||
"src.run_config.target = cpu_cluster.name\n",
|
||||
" \n",
|
||||
"run = experiment.submit(config=src)\n",
|
||||
"run"
|
||||
]
|
||||
|
||||
@@ -170,15 +170,15 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.runconfig import RunConfiguration\n",
|
||||
"from azureml.core import Environment\n",
|
||||
"\n",
|
||||
"# Editing a run configuration property on-fly.\n",
|
||||
"run_config_user_managed = RunConfiguration()\n",
|
||||
"user_managed_env = Environment(\"user-managed-env\")\n",
|
||||
"\n",
|
||||
"run_config_user_managed.environment.python.user_managed_dependencies = True\n",
|
||||
"user_managed_env.python.user_managed_dependencies = True\n",
|
||||
"\n",
|
||||
"# You can choose a specific Python environment by pointing to a Python path \n",
|
||||
"#run_config.environment.python.interpreter_path = '/home/johndoe/miniconda3/envs/myenv/bin/python'"
|
||||
"#user_managed_env.python.interpreter_path = '/home/johndoe/miniconda3/envs/myenv/bin/python'"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -197,7 +197,16 @@
|
||||
"source": [
|
||||
"from azureml.core import ScriptRunConfig\n",
|
||||
"\n",
|
||||
"src = ScriptRunConfig(source_directory='./', script='train.py', run_config=run_config_user_managed)\n",
|
||||
"src = ScriptRunConfig(source_directory='./', script='train.py')\n",
|
||||
"src.run_config.environment = user_managed_env"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"run = exp.submit(src)"
|
||||
]
|
||||
},
|
||||
@@ -277,13 +286,13 @@
|
||||
"source": [
|
||||
"from azureml.core.conda_dependencies import CondaDependencies\n",
|
||||
"\n",
|
||||
"run_config_system_managed = RunConfiguration()\n",
|
||||
"system_managed_env = Environment(\"system-managed-env\")\n",
|
||||
"\n",
|
||||
"run_config_system_managed.environment.python.user_managed_dependencies = False\n",
|
||||
"system_managed_env.python.user_managed_dependencies = False\n",
|
||||
"\n",
|
||||
"# Specify conda dependencies with scikit-learn\n",
|
||||
"cd = CondaDependencies.create(conda_packages=['scikit-learn'])\n",
|
||||
"run_config_system_managed.environment.python.conda_dependencies = cd"
|
||||
"system_managed_env.python.conda_dependencies = cd"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -302,7 +311,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"src = ScriptRunConfig(source_directory=\"./\", script='train.py', run_config=run_config_system_managed)\n",
|
||||
"src.run_config.environment = system_managed_env\n",
|
||||
"run = exp.submit(src)"
|
||||
]
|
||||
},
|
||||
@@ -371,18 +380,16 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"run_config_docker = RunConfiguration()\n",
|
||||
"run_config_docker.environment.python.user_managed_dependencies = False\n",
|
||||
"run_config_docker.environment.docker.enabled = True\n",
|
||||
"docker_env = Environment(\"docker-env\")\n",
|
||||
"\n",
|
||||
"docker_env.python.user_managed_dependencies = False\n",
|
||||
"docker_env.docker.enabled = True\n",
|
||||
"\n",
|
||||
"# use the default CPU-based Docker image from Azure ML\n",
|
||||
"run_config_docker.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE # Reference Docker image\n",
|
||||
"print(docker_env.docker.base_image)\n",
|
||||
"\n",
|
||||
"# Specify conda dependencies with scikit-learn\n",
|
||||
"cd = CondaDependencies.create(conda_packages=['scikit-learn'])\n",
|
||||
"run_config_docker.environment.python.conda_dependencies = cd\n",
|
||||
"\n",
|
||||
"src = ScriptRunConfig(source_directory=\"./\", script='train.py', run_config=run_config_docker)"
|
||||
"docker_env.python.conda_dependencies = cd"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -402,6 +409,8 @@
|
||||
"source": [
|
||||
"import subprocess\n",
|
||||
"\n",
|
||||
"src.run_config.environment = docker_env\n",
|
||||
"\n",
|
||||
"# Check if Docker is installed and Linux containers are enabled\n",
|
||||
"if subprocess.run(\"docker -v\", shell=True).returncode == 0:\n",
|
||||
" out = subprocess.check_output(\"docker system info\", shell=True).decode('ascii')\n",
|
||||
@@ -657,7 +666,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.8"
|
||||
"version": "3.6.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@@ -280,20 +280,11 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.runconfig import RunConfiguration\n",
|
||||
"from azureml.core import Environment\n",
|
||||
"from azureml.core.conda_dependencies import CondaDependencies\n",
|
||||
"\n",
|
||||
"# create a new RunConfig object\n",
|
||||
"conda_run_config = RunConfiguration(framework=\"python\")\n",
|
||||
"\n",
|
||||
"# Set compute target to the Linux DSVM\n",
|
||||
"conda_run_config.target = attached_dsvm_compute.name\n",
|
||||
"\n",
|
||||
"# set the data reference of the run configuration\n",
|
||||
"conda_run_config.data_references = {ds.name: dr}\n",
|
||||
"\n",
|
||||
"# specify CondaDependencies obj\n",
|
||||
"conda_run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
|
||||
"conda_env = Environment(\"conda-env\")\n",
|
||||
"conda_env.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -306,18 +297,14 @@
|
||||
"\n",
|
||||
"src = ScriptRunConfig(source_directory=script_folder, \n",
|
||||
" script='train.py', \n",
|
||||
" run_config=conda_run_config, \n",
|
||||
" # pass the datastore reference as a parameter to the training script\n",
|
||||
" arguments=['--data-folder', str(ds.as_download())] \n",
|
||||
" ) \n",
|
||||
"run = exp.submit(config=src)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note: if you need to cancel a run, you can follow [these instructions](https://aka.ms/aml-docs-cancel-run)."
|
||||
"\n",
|
||||
"src.run_config.framework = \"python\"\n",
|
||||
"src.run_config.environment = conda_env\n",
|
||||
"src.run_config.target = attached_dsvm_compute.name\n",
|
||||
"src.run_config.data_references = {ds.name: dr}"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -326,9 +313,18 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"run = exp.submit(config=src)\n",
|
||||
"\n",
|
||||
"run.wait_for_completion(show_output=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note: if you need to cancel a run, you can follow [these instructions](https://aka.ms/aml-docs-cancel-run)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -359,17 +355,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# create a new RunConfig object\n",
|
||||
"vm_run_config = RunConfiguration(framework=\"python\")\n",
|
||||
"\n",
|
||||
"# Set compute target to the Linux DSVM\n",
|
||||
"vm_run_config.target = attached_dsvm_compute.name\n",
|
||||
"\n",
|
||||
"# set the data reference of the run coonfiguration\n",
|
||||
"conda_run_config.data_references = {ds.name: dr}\n",
|
||||
"\n",
|
||||
"# Let system know that you will configure the Python environment yourself.\n",
|
||||
"vm_run_config.environment.python.user_managed_dependencies = True"
|
||||
"conda_env.python.user_managed_dependencies = True"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -385,11 +371,6 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"src = ScriptRunConfig(source_directory=script_folder, \n",
|
||||
" script='train.py', \n",
|
||||
" run_config=vm_run_config,\n",
|
||||
" # pass the datastore reference as a parameter to the training script\n",
|
||||
" arguments=['--data-folder', str(ds.as_download())])\n",
|
||||
"run = exp.submit(config=src)\n",
|
||||
"run.wait_for_completion(show_output=True)"
|
||||
]
|
||||
@@ -398,7 +379,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can choose to SSH into the VM and install Azure ML SDK, and any other missing dependencies, in that Python environment. For demonstration purposes, we simply are going to use another script `train2.py` that doesn't have azureml dependencies, and submit it instead."
|
||||
"You can choose to SSH into the VM and install Azure ML SDK, and any other missing dependencies, in that Python environment. For demonstration purposes, we simply are going to use another script `train2.py` that doesn't have azureml or data store dependencies, and submit it instead."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -411,7 +392,10 @@
|
||||
"shutil.copy('./train2.py', os.path.join(script_folder, 'train2.py'))\n",
|
||||
"\n",
|
||||
"with open(os.path.join(script_folder, './train2.py'), 'r') as training_script:\n",
|
||||
" print(training_script.read())"
|
||||
" print(training_script.read())\n",
|
||||
" \n",
|
||||
"src.run_config.data_references = {}\n",
|
||||
"src.script = \"train2.py\""
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -427,10 +411,8 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"src = ScriptRunConfig(source_directory=script_folder, \n",
|
||||
" script='train2.py', \n",
|
||||
" run_config=vm_run_config)\n",
|
||||
"run = exp.submit(config=src)\n",
|
||||
"\n",
|
||||
"run.wait_for_completion(show_output=True)"
|
||||
]
|
||||
},
|
||||
@@ -464,24 +446,10 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load the \"cpu-dsvm.runconfig\" file (created by the above attach operation) in memory\n",
|
||||
"docker_run_config = RunConfiguration(framework=\"python\")\n",
|
||||
"conda_env.docker.enabled = True\n",
|
||||
"conda_env.python.user_managed_dependencies = False\n",
|
||||
"\n",
|
||||
"# Set compute target to the Linux DSVM\n",
|
||||
"docker_run_config.target = attached_dsvm_compute.name\n",
|
||||
"\n",
|
||||
"# Use Docker in the remote VM\n",
|
||||
"docker_run_config.environment.docker.enabled = True\n",
|
||||
"\n",
|
||||
"# Use CPU base image from DockerHub\n",
|
||||
"docker_run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE\n",
|
||||
"print('Base Docker image is:', docker_run_config.environment.docker.base_image)\n",
|
||||
"\n",
|
||||
"# set the data reference of the run coonfiguration\n",
|
||||
"docker_run_config.data_references = {ds.name: dr}\n",
|
||||
"\n",
|
||||
"# specify CondaDependencies obj\n",
|
||||
"docker_run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])"
|
||||
"print('Base Docker image is:', conda_env.docker.base_image)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -498,20 +466,11 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"src = ScriptRunConfig(source_directory=script_folder, \n",
|
||||
" script='train.py', \n",
|
||||
" run_config=docker_run_config,\n",
|
||||
" # pass the datastore reference as a parameter to the training script\n",
|
||||
" arguments=['--data-folder', str(ds.as_download())])\n",
|
||||
"run = exp.submit(config=src)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"src.script = \"train.py\"\n",
|
||||
"src.run_config.data_references = {ds.name: dr}\n",
|
||||
"\n",
|
||||
"run = exp.submit(config=src)\n",
|
||||
"\n",
|
||||
"run.wait_for_completion(show_output=True)"
|
||||
]
|
||||
},
|
||||
@@ -525,20 +484,20 @@
|
||||
"\n",
|
||||
"```python\n",
|
||||
"# use an image available in Docker Hub without authentication\n",
|
||||
"run_config_docker.environment.docker.base_image = \"continuumio/miniconda3\"\n",
|
||||
"conda_env.docker.base_image = \"continuumio/miniconda3\"\n",
|
||||
"\n",
|
||||
"# or, use an image available in a private Azure Container Registry\n",
|
||||
"run_config_docker.environment.docker.base_image = \"mycustomimage:1.0\"\n",
|
||||
"run_config_docker.environment.docker.base_image_registry.address = \"myregistry.azurecr.io\"\n",
|
||||
"run_config_docker.environment.docker.base_image_registry.username = \"username\"\n",
|
||||
"run_config_docker.environment.docker.base_image_registry.password = \"password\"\n",
|
||||
"conda_env.docker.base_image = \"mycustomimage:1.0\"\n",
|
||||
"conda_env.docker.base_image_registry.address = \"myregistry.azurecr.io\"\n",
|
||||
"conda_env.docker.base_image_registry.username = \"username\"\n",
|
||||
"conda_env.docker.base_image_registry.password = \"password\"\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"When you are using a custom Docker image, you might already have your environment setup properly in a Python environment in the Docker image. In that case, you can skip specifying conda dependencies, and just use `user_managed_dependencies` option instead:\n",
|
||||
"```python\n",
|
||||
"run_config_docker.environment.python.user_managed_dependencies = True\n",
|
||||
"conda_env.python.user_managed_dependencies = True\n",
|
||||
"# path to the Python environment in the custom Docker image\n",
|
||||
"run_config.environment.python.interpreter_path = '/opt/conda/bin/python'\n",
|
||||
"conda_env.python.interpreter_path = '/opt/conda/bin/python'\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
@@ -640,7 +599,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.6"
|
||||
"version": "3.6.5"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
9
how-to-use-azureml/work-with-data/README.md
Normal file
9
how-to-use-azureml/work-with-data/README.md
Normal file
@@ -0,0 +1,9 @@
|
||||
# Work With Data Using Azure Machine Learning Service
|
||||
|
||||
Azure Machine Learning Datasets (preview) make it easier to access and work with your data. Datasets manage data in various scenarios such as model training and pipeline creation. Using the Azure Machine Learning SDK, you can access underlying storage, explore and prepare data, manage the life cycle of different Dataset definitions, and compare between Datasets used in training and in production.
|
||||
|
||||
- For an example of using Datasets, see the [sample](datasets).
|
||||
- For advanced data preparation examples, see [dataprep](dataprep).
|
||||
|
||||
|
||||
|
||||
@@ -222,3 +222,5 @@ Bug fixes
|
||||
IMPORTANT: Please read the notice and find out more about this NYC Taxi and Limousine Commission dataset here: http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml
|
||||
|
||||
IMPORTANT: Please read the notice and find out more about this Chicago Police Department dataset here: https://catalog.data.gov/dataset/crimes-2001-to-present-398a4
|
||||
|
||||
|
||||
|
||||
@@ -477,6 +477,13 @@
|
||||
"dflow_path = path.join(mkdtemp(), \"new_york_taxi.dprep\")\n",
|
||||
"combined_df.save(file_path=dflow_path)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -97,6 +97,13 @@
|
||||
"spark_df = df.take(5).to_pandas_dataframe()\n",
|
||||
"spark_df.head(5)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
1001
how-to-use-azureml/work-with-data/dataprep/data/crime-full.csv
Normal file
1001
how-to-use-azureml/work-with-data/dataprep/data/crime-full.csv
Normal file
File diff suppressed because it is too large
Load Diff
@@ -404,6 +404,13 @@
|
||||
"* [Sample your data](../../how-to-guides/subsetting-sampling.ipynb)\n",
|
||||
"* [Reference and link between Dataflows](../../how-to-guides/join.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -152,4 +152,8 @@ For an end-to-end tutorial, you may refer to [Dataset tutorial](datasets-tutoria
|
||||
- Register the Dataset in your workspace for easy access in training.
|
||||
- Take snapshots of data to ensure models can be trained with the same data every time.
|
||||
- Use registered Dataset in your training script.
|
||||
- Create and use multiple Dataset definitions to ensure that updates to the definition don't break existing pipelines/scripts.
|
||||
- Create and use multiple Dataset definitions to ensure that updates to the definition don't break existing pipelines/scripts.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -15,22 +15,25 @@ from sklearn.tree import DecisionTreeClassifier
|
||||
run = Run.get_context()
|
||||
workspace = run.experiment.workspace
|
||||
|
||||
dataset_name = 'training_data'
|
||||
dataset_name = 'clean_Titanic_tutorial'
|
||||
|
||||
snapshot_name = 'train_snapshot'
|
||||
|
||||
dataset = Dataset.get(workspace=workspace, name=dataset_name)
|
||||
dflow = dataset.get_definition()
|
||||
dflow_val, dflow_train = dflow.random_split(percentage=0.3)
|
||||
df = dataset.get_snapshot(snapshot_name=snapshot_name).to_pandas_dataframe()
|
||||
|
||||
y_df = dflow_train.keep_columns(['HasDetections']).to_pandas_dataframe()
|
||||
x_df = dflow_train.drop_columns(['HasDetections']).to_pandas_dataframe()
|
||||
y_val = dflow_val.keep_columns(['HasDetections']).to_pandas_dataframe()
|
||||
x_val = dflow_val.drop_columns(['HasDetections']).to_pandas_dataframe()
|
||||
x_col = ['Pclass', 'Sex', 'SibSp', 'Parch']
|
||||
y_col = ['Survived']
|
||||
x_df = df.loc[:, x_col]
|
||||
y_df = df.loc[:, y_col]
|
||||
|
||||
data = {"train": {"X": x_df, "y": y_df},
|
||||
x_train, x_test, y_train, y_test = train_test_split(x_df, y_df, test_size=0.2, random_state=223)
|
||||
|
||||
"validation": {"X": x_val, "y": y_val}}
|
||||
data = {"train": {"X": x_train, "y": y_train},
|
||||
|
||||
"test": {"X": x_test, "y": y_test}}
|
||||
|
||||
clf = DecisionTreeClassifier().fit(data["train"]["X"], data["train"]["y"])
|
||||
|
||||
print('Accuracy of Decision Tree classifier on training set: {:.2f}'.format(clf.score(x_df, y_df)))
|
||||
print('Accuracy of Decision Tree classifier on validation set: {:.2f}'.format(clf.score(x_val, y_val)))
|
||||
print('Accuracy of Decision Tree classifier on training set: {:.2f}'.format(clf.score(x_train, y_train)))
|
||||
print('Accuracy of Decision Tree classifier on test set: {:.2f}'.format(clf.score(x_test, y_test)))
|
||||
|
||||
52
index.html
52
index.html
@@ -1,52 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
|
||||
<meta name="google-site-verification" content="fkZxAt5AEHiB_Wom2R_25VTmNyj19J8lZlfTREsaEN4" />
|
||||
|
||||
<title>Azure Machine Learning</title>
|
||||
|
||||
</head>
|
||||
<body>
|
||||
<h1 id="azure-machine-learning-service-example-notebooks">Azure Machine Learning service example notebooks</h1>
|
||||
<p>This repository contains example notebooks demonstrating the <a href="https://azure.microsoft.com/en-us/services/machine-learning-service/">Azure Machine Learning</a> Python SDK which allows you to build, train, deploy and manage machine learning solutions using Azure. The AML SDK allows you the choice of using local or cloud compute resources, while managing and maintaining the complete data science workflow from the cloud.</p>
|
||||
<div class="figure">
|
||||
|
||||
<img src="https://raw.githubusercontent.com/MicrosoftDocs/azure-docs/master/articles/machine-learning/service/media/overview-what-is-azure-ml/aml.png" alt="Azure ML workflow" /><p class="caption">Azure ML workflow</p>
|
||||
</div>
|
||||
<h2 id="quick-installation">Quick installation</h2>
|
||||
<pre class="sh"><code>pip install azureml-sdk</code></pre>
|
||||
<p>Read more detailed instructions on <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/NBSETUP.md">how to set up your environment</a> using Azure Notebook service, your own Jupyter notebook server, or Docker.</p>
|
||||
<h2 id="how-to-navigate-and-use-the-example-notebooks">How to navigate and use the example notebooks?</h2>
|
||||
<p>You should always run the <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/configuration.ipynb">Configuration</a> notebook first when setting up a notebook library on a new machine or in a new environment. It configures your notebook library to connect to an Azure Machine Learning workspace, and sets up your workspace and compute to be used by many of the other examples.</p>
|
||||
<p>If you want to...</p>
|
||||
<ul>
|
||||
<li>...try out and explore Azure ML, start with image classification tutorials: <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/tutorials/img-classification-part1-training.ipynb">Part 1 (Training)</a> and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/tutorials/img-classification-part2-deploy.ipynb">Part 2 (Deployment)</a>.</li>
|
||||
<li>...prepare your data and do automated machine learning, start with regression tutorials: <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/tutorials/regression-part1-data-prep.ipynb">Part 1 (Data Prep)</a> and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/tutorials/regression-part2-automated-ml.ipynb">Part 2 (Automated ML)</a>.</li>
|
||||
<li>...learn about experimentation and tracking run history, first <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-within-notebook/train-within-notebook.ipynb">train within Notebook</a>, then try <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-on-remote-vm/train-on-remote-vm.ipynb">training on remote VM</a> and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/logging-api/logging-api.ipynb">using logging APIs</a>.</li>
|
||||
<li>...train deep learning models at scale, first learn about <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-on-amlcompute/train-on-amlcompute.ipynb">Machine Learning Compute</a>, and then try <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training-with-deep-learning/train-hyperparameter-tune-deploy-with-pytorch/train-hyperparameter-tune-deploy-with-pytorch.ipynb">distributed hyperparameter tuning</a> and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training-with-deep-learning/distributed-pytorch-with-horovod/distributed-pytorch-with-horovod.ipynb">distributed training</a>.</li>
|
||||
<li>...deploy models as a realtime scoring service, first learn the basics by <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-within-notebook/train-within-notebook.ipynb">training within Notebook and deploying to Azure Container Instance</a>, then learn how to <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/register-model-create-image-deploy-service/register-model-create-image-deploy-service.ipynb">register and manage models, and create Docker images</a>, and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/production-deploy-to-aks/production-deploy-to-aks.ipynb">production deploy models on Azure Kubernetes Cluster</a>.</li>
|
||||
<li>...deploy models as a batch scoring service, first <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-within-notebook/train-within-notebook.ipynb">train a model within Notebook</a>, learn how to <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/register-model-create-image-deploy-service/register-model-create-image-deploy-service.ipynb">register and manage models</a>, then <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-on-amlcompute/train-on-amlcompute.ipynb">create Machine Learning Compute for scoring compute</a>, and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/pipeline-mpi-batch-prediction.ipynb">use Machine Learning Pipelines to deploy your model</a>.</li>
|
||||
<li>...monitor your deployed models, learn about using <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/enable-app-insights-in-production-service/enable-app-insights-in-production-service.ipynb">App Insights</a> and <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/enable-data-collection-for-models-in-aks/enable-data-collection-for-models-in-aks.ipynb">model data collection</a>.</li>
|
||||
</ul>
|
||||
<h2 id="tutorials">Tutorials</h2>
|
||||
<p>The <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/tutorials">Tutorials</a> folder contains notebooks for the tutorials described in the <a href="https://aka.ms/aml-docs">Azure Machine Learning documentation</a>.</p>
|
||||
<h2 id="how-to-use-azure-ml">How to use Azure ML</h2>
|
||||
<p>The <a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml">How to use Azure ML</a> folder contains specific examples demonstrating the features of the Azure Machine Learning SDK</p>
|
||||
<ul>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training">Training</a> - Examples of how to build models using Azure ML's logging and execution capabilities on local and remote compute targets</li>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training-with-deep-learning">Training with Deep Learning</a> - Examples demonstrating how to build deep learning models using estimators and parameter sweeps</li>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/manage-azureml-service">Manage Azure ML Service</a> - Examples how to perform tasks, such as authenticate against Azure ML service in different ways.</li>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/automated-machine-learning">Automated Machine Learning</a> - Examples using Automated Machine Learning to automatically generate optimal machine learning pipelines and models</li>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines">Machine Learning Pipelines</a> - Examples showing how to create and use reusable pipelines for training and batch scoring</li>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment">Deployment</a> - Examples showing how to deploy and manage machine learning models and solutions</li>
|
||||
<li><a href="https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/azure-databricks">Azure Databricks</a> - Examples showing how to use Azure ML with Azure Databricks</li>
|
||||
</ul>
|
||||
<h2 id="projects-using-azure-machine-learning">Projects using Azure Machine Learning</h2>
|
||||
<p>Visit following repos to see projects contributed by Azure ML users:</p>
|
||||
<ul>
|
||||
<li><a href="https://github.com/Microsoft/AzureML-BERT">Fine tune natural language processing models using Azure Machine Learning service</a></li>
|
||||
<li><a href="https://github.com/amynic/azureml-sdk-fashion">Fashion MNIST with Azure ML SDK</a></li>
|
||||
</ul>
|
||||
</body>
|
||||
</html>
|
||||
52
pr.md
52
pr.md
@@ -1,52 +0,0 @@
|
||||
# Azure Machine Learning Resources & Links
|
||||
## Product Documentation
|
||||
- [Azure Machine Learning service](https://docs.microsoft.com/en-us/azure/machine-learning/service/)
|
||||
- [Azure Machine Learning Studio](https://docs.microsoft.com/en-us/azure/machine-learning/studio/)
|
||||
|
||||
## Product Team Blogs
|
||||
- [What’s new in Azure Machine Learning service](https://aka.ms/aml-blog-whats-new)
|
||||
- [Announcing automated ML capability in Azure Machine Learning](https://aka.ms/aml-blog-automl)
|
||||
- [Experimentation using Azure Machine Learning](https://aka.ms/aml-blog-experimentation)
|
||||
- [Azure AI – Making AI real for business](https://aka.ms/aml-blog-overview)
|
||||
|
||||
## Community Blogs
|
||||
- [Power Bat – How Spektacom is Powering the Game of Cricket with Microsoft AI](https://blogs.technet.microsoft.com/machinelearning/2018/10/11/power-bat-how-spektacom-is-powering-the-game-of-cricket-with-microsoft-ai/)
|
||||
|
||||
## Ignite 2018 Public Preview Launch Sessions
|
||||
- [AI with Azure Machine Learning services: Simplifying the data science process](https://myignite.techcommunity.microsoft.com/sessions/66248)
|
||||
- [AI TechTalk: Azure Machine Learning SDK - a walkthrough](https://myignite.techcommunity.microsoft.com/sessions/66265)
|
||||
- [AI for an intelligent cloud and intelligent edge: Discover, deploy, and manage with Azure ML services](https://myignite.techcommunity.microsoft.com/sessions/65389)
|
||||
- [Generating high quality models efficiently using Automated ML and Hyperparameter Tuning](https://myignite.techcommunity.microsoft.com/sessions/66245)
|
||||
- [AI for pros: Deep learning with PyTorch using the Azure Data Science Virtual Machine and scaling training with Azure ML](https://myignite.techcommunity.microsoft.com/sessions/66244)
|
||||
|
||||
## Get-started Videos on YouTube
|
||||
- [Get started with Python SDK](https://youtu.be/VIsXeTuW3FU)
|
||||
- [Get started from Azure Portal](https://youtu.be/lCkYUHV86Mk)
|
||||
|
||||
|
||||
## Third Party Articles
|
||||
- [Azure’s new machine learning features embrace Python](https://www.infoworld.com/article/3306840/azure/azures-new-machine-learning-features-embrace-python.html) (InfoWorld)
|
||||
- [How to use Azure ML in Windows 10](https://www.infoworld.com/article/3308381/azure/how-to-use-azure-ml-in-windows-10.html) (InfoWorld)
|
||||
- [How Azure ML Streamlines Cloud-based Machine Learning](https://thenewstack.io/how-the-azure-ml-streamlines-cloud-based-machine-learning/) (The New Stack)
|
||||
- [Facebook launches PyTorch 1.0 with integrations for Google Cloud, AWS, and Azure Machine Learning](https://venturebeat.com/2018/10/02/facebook-launches-pytorch-1-0-integrations-for-google-cloud-aws-and-azure-machine-learning/) (VentureBeat)
|
||||
- [How Microsoft Uses Machine Learning to Help You Build Machine Learning Pipelines](https://towardsdatascience.com/how-microsoft-uses-machine-learning-to-help-you-build-machine-learning-pipelines-be75f710613b) (Towards Data Science)
|
||||
- [Microsoft's Machine Learning Tools for Developers Get Smarter](https://techcrunch.com/2018/09/24/microsofts-machine-learning-tools-for-developers-get-smarter/) (TechCrunch)
|
||||
- [Microsoft introduces Azure service to automatically build AI models](https://venturebeat.com/2018/09/24/microsoft-introduces-azure-service-to-automatically-build-ai-models/) (VentureBeat)
|
||||
|
||||
## Community Projects
|
||||
- [Use Papermill with Azure ML](https://github.com/jreynolds01/papermill_execution_azureml/)
|
||||
- [Fashion MNIST](https://github.com/amynic/azureml-sdk-fashion)
|
||||
- Keras on Databricks
|
||||
- [Samples from CSS](https://github.com/Azure/AMLSamples)
|
||||
|
||||
|
||||
## Azure Machine Learning Studio Resources
|
||||
- [A-Z Machine Learning using Azure Machine Learning (AzureML)](https://www.udemy.com/machine-learning-using-azureml/)
|
||||
- [Machine Learning In The Cloud With Azure Machine Learning](https://www.udemy.com/machine-learning-in-the-cloud-with-azure-machine-learning/)
|
||||
- [How to Become A Data Scientist Using Azure Machine Learning](https://www.udemy.com/azure-machine-learning-introduction/)
|
||||
- [Learn Azure Machine Learning from scratch](https://www.udemy.com/learn-azure-machine-learning-from-scratch/)
|
||||
- [Azure Machine Learning Studio PowerShell Module](https://aka.ms/amlps)
|
||||
|
||||
## Forum Help
|
||||
- [Azure Machine Learning service](https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureMachineLearningService)
|
||||
- [Azure Machine Learning Studio](https://social.msdn.microsoft.com/forums/azure/en-US/home?forum=MachineLearning)
|
||||
@@ -18,3 +18,5 @@ If you are using an Azure Machine Learning Notebook VM, you are all set. Otherwi
|
||||
* [Part 2](regression-part2-automated-ml.ipynb): Train a model using Automated Machine Learning.
|
||||
|
||||
Also find quickstarts and how-tos on the [official documentation site for Azure Machine Learning service](https://docs.microsoft.com/en-us/azure/machine-learning/service/).
|
||||
|
||||
|
||||
@@ -626,6 +626,13 @@
|
||||
"\n",
|
||||
"> [Tutorial 2 - Deploy models](img-classification-part2-deploy.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -587,6 +587,13 @@
|
||||
" \n",
|
||||
"You can also try out the [regression tutorial](regression-part1-data-prep.ipynb)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -599,6 +599,13 @@
|
||||
"\n",
|
||||
"> [Tutorial #2: Train regression model](regression-part2-automated-ml.ipynb)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
"> * Run the model locally with custom parameters\n",
|
||||
"> * Explore the results\n",
|
||||
"\n",
|
||||
"If you don\u00e2\u20ac\u2122t have an Azure subscription, create a [free account](https://aka.ms/AMLfree) before you begin. \n",
|
||||
"If you do not have an Azure subscription, create a [free account](https://aka.ms/AMLfree) before you begin. \n",
|
||||
"\n",
|
||||
"> Code in this article was tested with Azure Machine Learning SDK version 1.0.0\n",
|
||||
"\n",
|
||||
@@ -485,7 +485,7 @@
|
||||
">The resources you created can be used as prerequisites to other Azure Machine Learning service tutorials and how-to articles. \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"If you don't plan to use the resources you created, delete them, so you don't incur any charges:\n",
|
||||
"If you do not plan to use the resources you created, delete them, so you do not incur any charges:\n",
|
||||
"\n",
|
||||
"1. In the Azure portal, select **Resource groups** on the far left.\n",
|
||||
"\n",
|
||||
@@ -510,6 +510,13 @@
|
||||
"\n",
|
||||
"[Deploy your model](https://docs.microsoft.com/azure/machine-learning/service/tutorial-deploy-models-with-aml) with Azure Machine Learning."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
|
||||
Reference in New Issue
Block a user