mirror of
https://github.com/Azure/MachineLearningNotebooks.git
synced 2025-12-19 17:17:04 -05:00
245 lines
8.7 KiB
Plaintext
245 lines
8.7 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
|
"\n",
|
|
"Licensed under the MIT License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Azure ML Reinforcement Learning Sample - Setting Up Development Environment\n",
|
|
"\n",
|
|
"Ray multi-node cluster setup requires all worker nodes to be able to communicate with the head node. This notebook explains you how to setup a virtual network, to be used by the Ray head and worker compute targets, created and used in other notebook examples."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Prerequisite\n",
|
|
"\n",
|
|
"The user should have completed the Azure Machine Learning Tutorial: [Get started creating your first ML experiment with the Python SDK](https://docs.microsoft.com/en-us/azure/machine-learning/tutorial-1st-experiment-sdk-setup). You will need to make sure that you have a valid subscription id, a resource group and a workspace."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Azure Machine Learning SDK \n",
|
|
"Display the Azure Machine Learning SDK version."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Azure ML Core imports\n",
|
|
"import azureml.core\n",
|
|
"\n",
|
|
"# Check core SDK version number\n",
|
|
"print(\"Azure ML SDK Version: \", azureml.core.VERSION)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Get Azure ML workspace\n",
|
|
"Get a reference to an existing Azure ML workspace. Please make sure that the VM sizes `STANDARD_NC6` and `STANDARD_D2_V2` are supported in the workspace's region.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core import Workspace\n",
|
|
"\n",
|
|
"ws = Workspace.from_config()\n",
|
|
"print(ws.name, ws.location, ws.resource_group, sep = ' | ')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Create Virtual Network\n",
|
|
"\n",
|
|
"If you are using separate compute targets for the Ray head and worker, a virtual network must be created in the resource group. If you have alraeady created a virtual network in the resource group, you can skip this step.\n",
|
|
"\n",
|
|
"To do this, you first must install the Azure Networking API.\n",
|
|
"\n",
|
|
"`pip install --upgrade azure-mgmt-network`"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# If you need to install the Azure Networking SDK, uncomment the following line.\n",
|
|
"#!pip install --upgrade azure-mgmt-network"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azure.mgmt.network import NetworkManagementClient\n",
|
|
"\n",
|
|
"# Virtual network name\n",
|
|
"vnet_name =\"your_vnet\"\n",
|
|
"\n",
|
|
"# Default subnet\n",
|
|
"subnet_name =\"default\"\n",
|
|
"\n",
|
|
"# The Azure subscription you are using\n",
|
|
"subscription_id=ws.subscription_id\n",
|
|
"\n",
|
|
"# The resource group for the RL cluster\n",
|
|
"resource_group=ws.resource_group\n",
|
|
"\n",
|
|
"# Azure region of the resource group\n",
|
|
"location=ws.location\n",
|
|
"\n",
|
|
"network_client = NetworkManagementClient(ws._auth_object, subscription_id)\n",
|
|
"\n",
|
|
"async_vnet_creation = network_client.virtual_networks.create_or_update(\n",
|
|
" resource_group,\n",
|
|
" vnet_name,\n",
|
|
" {\n",
|
|
" 'location': location,\n",
|
|
" 'address_space': {\n",
|
|
" 'address_prefixes': ['10.0.0.0/16']\n",
|
|
" }\n",
|
|
" }\n",
|
|
")\n",
|
|
"\n",
|
|
"async_vnet_creation.wait()\n",
|
|
"print(\"VNet created successfully: \", async_vnet_creation.result())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Set up Network Security Group on Virtual Network\n",
|
|
"\n",
|
|
"Depending on your Azure setup, you may need to open certain ports to make it possible for Azure to manage the compute targets that you create. The ports that need to be opened are described [here](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-enable-virtual-network).\n",
|
|
"\n",
|
|
"A common situation is that ports `29876-29877` are closed. The following code will add a security rule to open these ports. Or you can do this manually in the [Azure portal](https://portal.azure.com).\n",
|
|
"\n",
|
|
"You may need to modify the code below to match your scenario."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import azure.mgmt.network.models\n",
|
|
"\n",
|
|
"security_group_name = vnet_name + '-' + \"nsg\"\n",
|
|
"security_rule_name = \"AllowAML\"\n",
|
|
"\n",
|
|
"# Create a network security group\n",
|
|
"nsg_params = azure.mgmt.network.models.NetworkSecurityGroup(\n",
|
|
" location=location,\n",
|
|
" security_rules=[\n",
|
|
" azure.mgmt.network.models.SecurityRule(\n",
|
|
" name=security_rule_name,\n",
|
|
" access=azure.mgmt.network.models.SecurityRuleAccess.allow,\n",
|
|
" description='Azure ML RL rule',\n",
|
|
" destination_address_prefix='*',\n",
|
|
" destination_port_range='29876-29877',\n",
|
|
" direction=azure.mgmt.network.models.SecurityRuleDirection.inbound,\n",
|
|
" priority=400,\n",
|
|
" protocol=azure.mgmt.network.models.SecurityRuleProtocol.tcp,\n",
|
|
" source_address_prefix='BatchNodeManagement',\n",
|
|
" source_port_range='*'\n",
|
|
" ),\n",
|
|
" ],\n",
|
|
")\n",
|
|
"\n",
|
|
"async_nsg_creation = network_client.network_security_groups.create_or_update(\n",
|
|
" resource_group,\n",
|
|
" security_group_name,\n",
|
|
" nsg_params,\n",
|
|
")\n",
|
|
"\n",
|
|
"async_nsg_creation.wait() \n",
|
|
"print(\"Network security group created successfully: \", async_nsg_creation.result())\n",
|
|
"\n",
|
|
"network_security_group = network_client.network_security_groups.get(\n",
|
|
" resource_group,\n",
|
|
" security_group_name,\n",
|
|
")\n",
|
|
"\n",
|
|
"# Define a subnet to be created with network security group\n",
|
|
"subnet = azure.mgmt.network.models.Subnet(\n",
|
|
" id='default',\n",
|
|
" address_prefix='10.0.0.0/24',\n",
|
|
" network_security_group=network_security_group\n",
|
|
" )\n",
|
|
" \n",
|
|
"# Create subnet on vnet\n",
|
|
"async_subnet_creation = network_client.subnets.create_or_update(\n",
|
|
" resource_group_name=resource_group,\n",
|
|
" virtual_network_name=vnet_name,\n",
|
|
" subnet_name=subnet_name,\n",
|
|
" subnet_parameters=subnet\n",
|
|
")\n",
|
|
"\n",
|
|
"async_subnet_creation.wait()\n",
|
|
"print(\"Subnet created successfully:\", async_subnet_creation.result())"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"authors": [
|
|
{
|
|
"name": "vineetg"
|
|
}
|
|
],
|
|
"kernelspec": {
|
|
"display_name": "Python 3.6",
|
|
"language": "python",
|
|
"name": "python36"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.5"
|
|
},
|
|
"notice": "Copyright (c) Microsoft Corporation. All rights reserved.\u00e2\u20ac\u00afLicensed under the MIT License.\u00e2\u20ac\u00af "
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
} |