mirror of
https://github.com/Azure/MachineLearningNotebooks.git
synced 2025-12-20 17:45:10 -05:00
543 lines
21 KiB
Plaintext
543 lines
21 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
|
"\n",
|
|
"Licensed under the MIT License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Authentication in Azure Machine Learning\n",
|
|
"\n",
|
|
"This notebook shows you how to authenticate to your Azure ML Workspace using\n",
|
|
"\n",
|
|
" 1. Interactive Login Authentication\n",
|
|
" 2. Azure CLI Authentication\n",
|
|
" 3. Managed Service Identity (MSI) Authentication\n",
|
|
" 4. Service Principal Authentication\n",
|
|
" 5. Token Authentication\n",
|
|
" \n",
|
|
"The interactive authentication is suitable for local experimentation on your own computer. Azure CLI authentication is suitable if you are already using Azure CLI for managing Azure resources, and want to sign in only once. The MSI and Service Principal authentication are suitable for automated workflows, for example as part of Azure Devops build."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core import Workspace"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Interactive Authentication\n",
|
|
"\n",
|
|
"Interactive authentication is the default mode when using Azure ML SDK.\n",
|
|
"\n",
|
|
"When you connect to your workspace using workspace.from_config, you will get an interactive login dialog."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"ws = Workspace.from_config()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Also, if you explicitly specify the subscription ID, resource group and workspace name, you will get the dialog."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"ws = Workspace(subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Note the user you're authenticated as must have access to the subscription and resource group. If you receive an error\n",
|
|
"\n",
|
|
"```\n",
|
|
"AuthenticationException: You don't have access to xxxxxx-xxxx-xxx-xxx-xxxxxxxxxx subscription. All the subscriptions that you have access to = ...\n",
|
|
"```\n",
|
|
"\n",
|
|
"check that the you used correct login and entered the correct subscription ID."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In some cases, you may see a version of the error message containing text: ```All the subscriptions that you have access to = []```\n",
|
|
"\n",
|
|
"In such a case, you may have to specify the tenant ID of the Azure Active Directory you're using. An example would be accessing a subscription as a guest to a tenant that is not your default. You specify the tenant by explicitly instantiating _InteractiveLoginAuthentication_ with Tenant ID as argument. The Tenant ID can be found, for example, from https://portal.azure.com under **Azure Active Directory**, **Properties** as Directory ID."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"sample-interactiveloginauth-tenantid"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core.authentication import InteractiveLoginAuthentication\n",
|
|
"\n",
|
|
"interactive_auth = InteractiveLoginAuthentication(tenant_id=\"my-tenant-id\")\n",
|
|
"\n",
|
|
"ws = Workspace(subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\",\n",
|
|
" auth=interactive_auth)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Despite having access to the workspace, you may sometimes see the following error when retrieving it:\n",
|
|
"\n",
|
|
"```\n",
|
|
"You are currently logged-in to xxxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxxxx tenant. You don't have access to xxxxxx-xxxx-xxx-xxx-xxxxxxxxxx subscription, please check if it is in this tenant.\n",
|
|
"```\n",
|
|
"\n",
|
|
"This error sometimes occurs when you are trying to access a subscription to which you were recently added. In this case, you need to force authentication again to avoid using a cached authentication token that has not picked up the new permissions. You can do so by setting `force=true` on the `InteractiveLoginAuthentication()` object's constructor as follows:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"forced_interactive_auth = InteractiveLoginAuthentication(tenant_id=\"my-tenant-id\", force=True)\n",
|
|
"\n",
|
|
"ws = Workspace(subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\",\n",
|
|
" auth=forced_interactive_auth)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Azure CLI Authentication\n",
|
|
"\n",
|
|
"If you have installed azure-cli package, and used ```az login``` command to log in to your Azure Subscription, you can use _AzureCliAuthentication_ class.\n",
|
|
"\n",
|
|
"Note that interactive authentication described above won't use existing Azure CLI auth tokens. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"sample-azurecliauth"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core.authentication import AzureCliAuthentication\n",
|
|
"\n",
|
|
"cli_auth = AzureCliAuthentication()\n",
|
|
"\n",
|
|
"ws = Workspace(subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\",\n",
|
|
" auth=cli_auth)\n",
|
|
"\n",
|
|
"print(\"Found workspace {} at location {}\".format(ws.name, ws.location))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### MSI Authentication\n",
|
|
"\n",
|
|
"__Note__: _MSI authentication is supported only when using SDK from Azure Virtual Machine. The code below will fail on local computer._\n",
|
|
"\n",
|
|
"When using Azure ML SDK on Azure Virtual Machine (VM), you can use Managed Service Identity (MSI) based authentication. This mode allows the VM connect to the Workspace without storing credentials in the Python code.\n",
|
|
"\n",
|
|
"As a prerequisite, enable System-assigned Managed Identity for your VM as described in [Configure managed identities for Azure resources on a VM using the Azure portal](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/qs-configure-portal-windows-vm).\n",
|
|
"\n",
|
|
"Then, assign the VM access to your Workspace. For example from Azure Portal, navigate to your workspace, select __Access Control (IAM)__, __Add Role Assignment__, specify __Virtual Machine__ for __Assign Access To__ dropdown, and select your VM's identity.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"After completing these steps, you can use authenticate using MsiAuthentication instance."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"sample-msiauth"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core.authentication import MsiAuthentication\n",
|
|
"\n",
|
|
"msi_auth = MsiAuthentication()\n",
|
|
"\n",
|
|
"ws = Workspace(subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\",\n",
|
|
" auth=msi_auth)\n",
|
|
"\n",
|
|
"print(\"Found workspace {} at location {}\".format(ws.name, ws.location))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Service Principal Authentication\n",
|
|
"\n",
|
|
"When setting up a machine learning workflow as an automated process, we recommend using Service Principal Authentication. This approach decouples the authentication from any specific user login, and allows managed access control.\n",
|
|
"\n",
|
|
"Note that you must have administrator privileges over the Azure subscription to complete these steps.\n",
|
|
"\n",
|
|
"The first step is to create a service principal. First, go to [Azure Portal](https://portal.azure.com), select **Azure Active Directory** and **App Registrations**. Then select **+New application**, give your service principal a name, for example _my-svc-principal_. You can leave other parameters as is.\n",
|
|
"\n",
|
|
"Then click **Register**.\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"From the page for your newly created service principal, copy the _Application ID_ and _Tenant ID_ as they are needed later.\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Then select **Certificates & secrets**, and **+New client secret** write a description for your key, and select duration. Then click **Add**, and copy the value of client secret to a secure location.\n",
|
|
"\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Finally, you need to give the service principal permissions to access your workspace. Navigate to **Resource Groups**, to the resource group for your Machine Learning Workspace. \n",
|
|
"\n",
|
|
"Then select **Access Control (IAM)** and **Add a role assignment**. For _Role_, specify which level of access you need to grant, for example _Contributor_. Start entering your service principal name and once it is found, select it, and click **Save**.\n",
|
|
"\n",
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now you are ready to use the service principal authentication. For example, to connect to your Workspace, see code below and enter your own values for tenant ID, application ID, subscription ID, resource group and workspace.\n",
|
|
"\n",
|
|
"**We strongly recommended that you do not insert the secret password to code**. Instead, you can use environment variables to pass it to your code, for example through Azure Key Vault, or through secret build variables in Azure DevOps. For local testing, you can for example use following PowerShell command to set the environment variable.\n",
|
|
"\n",
|
|
"```\n",
|
|
"$env:AZUREML_PASSWORD = \"my-password\"\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"sample-serviceprincipalauth-tenantid"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"from azureml.core.authentication import ServicePrincipalAuthentication\n",
|
|
"\n",
|
|
"svc_pr_password = os.environ.get(\"AZUREML_PASSWORD\")\n",
|
|
"\n",
|
|
"svc_pr = ServicePrincipalAuthentication(\n",
|
|
" tenant_id=\"my-tenant-id\",\n",
|
|
" service_principal_id=\"my-application-id\",\n",
|
|
" service_principal_password=svc_pr_password)\n",
|
|
"\n",
|
|
"\n",
|
|
"ws = Workspace(\n",
|
|
" subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\",\n",
|
|
" auth=svc_pr\n",
|
|
" )\n",
|
|
"\n",
|
|
"print(\"Found workspace {} at location {}\".format(ws.name, ws.location))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"See [Register an application with the Microsoft identity platform](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app) quickstart for more details about application registrations. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Token Authentication\n",
|
|
"\n",
|
|
"When token generation and its refresh needs to be outside on AML SDK, we recommend using Token Authentication. It can be used for getting token for AML or ARM audience. Thus giving more granular control over token generated.\n",
|
|
"\n",
|
|
"This authentication class requires users to provide method `get_token_for_audience` which will be called to retrieve the token based on the audience passed.\n",
|
|
"\n",
|
|
"Audience that is passed to `get_token_for_audience` can be ARM or AML. Exact value that will be passed as audience will depend on cloud and type for audience."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core.authentication import TokenAuthentication, Audience\n",
|
|
"\n",
|
|
"# This is a sample method to retrieve token and will be passed to TokenAuthentication\n",
|
|
"def get_token_for_audience(audience):\n",
|
|
" from adal import AuthenticationContext\n",
|
|
" client_id = \"my-client-id\"\n",
|
|
" client_secret = \"my-client-secret\"\n",
|
|
" tenant_id = \"my-tenant-id\"\n",
|
|
" auth_context = AuthenticationContext(\"https://login.microsoftonline.com/{}\".format(tenant_id))\n",
|
|
" resp = auth_context.acquire_token_with_client_credentials(audience,client_id,client_secret)\n",
|
|
" token = resp[\"accessToken\"]\n",
|
|
" return token\n",
|
|
"\n",
|
|
"\n",
|
|
"token_auth = TokenAuthentication(get_token_for_audience=get_token_for_audience)\n",
|
|
"\n",
|
|
"ws = Workspace(\n",
|
|
" subscription_id=\"my-subscription-id\",\n",
|
|
" resource_group=\"my-ml-rg\",\n",
|
|
" workspace_name=\"my-ml-workspace\",\n",
|
|
" auth=token_auth\n",
|
|
" )\n",
|
|
"\n",
|
|
"print(\"Found workspace {} at location {}\".format(ws.name, ws.location))\n",
|
|
"\n",
|
|
"token_aml_audience = token_auth.get_token(Audience.aml)\n",
|
|
"token_arm_audience = token_auth.get_token(Audience.arm)\n",
|
|
"\n",
|
|
"# Value of audience pass to `get_token_for_audience` can be retrieved as follows:\n",
|
|
"# aud_aml_val = token_auth.get_aml_resource_id() # For AML\n",
|
|
"# aud_arm_val = token_auth._cloud_type.endpoints.active_directory_resource_id # For ARM\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Token authentication object can be used to retrieve token for either AML or ARM audience,\n",
|
|
"which can be used by other clients to authenticate to AML or ARM"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Using Secrets in Remote Runs\n",
|
|
"\n",
|
|
"Sometimes, you may have to pass a secret to a remote run, for example username and password to authenticate against external data source.\n",
|
|
"\n",
|
|
"Azure ML SDK enables this use case through Key Vault associated with your workspace. The workflow for adding a secret is following.\n",
|
|
"\n",
|
|
"On local computer:\n",
|
|
"\n",
|
|
" 1. Read in a local secret, for example from environment variable or user input. To keep them secret, do not insert secret values into code as hard-coded strings.\n",
|
|
" 2. Obtain a reference to the keyvault\n",
|
|
" 3. Add the secret name-value pair in the key vault.\n",
|
|
" \n",
|
|
"The secret is then available for remote runs as shown further below.\n",
|
|
"\n",
|
|
"__Note__: The _azureml.core.keyvault.Keyvault_ is different from _azure.keyvault_ library. It is intended as simplified wrapper for setting, getting and listing user secrets in Workspace Key Vault."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"sample-keyvault"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import uuid\n",
|
|
"\n",
|
|
"local_secret = os.environ.get(\"LOCAL_SECRET\", default = str(uuid.uuid4())) # Use random UUID as a substitute for real secret.\n",
|
|
"keyvault = ws.get_default_keyvault()\n",
|
|
"keyvault.set_secret(name=\"secret-name\", value = local_secret)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"The _set_secret_ method adds a new secret if one doesn't exist, or updates an existing one with new value.\n",
|
|
"\n",
|
|
"You can list secret names you've added. This method doesn't return the values of the secrets."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"keyvault.list_secrets()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can retrieve the value of the secret, and validate that it matches the original value. \n",
|
|
"\n",
|
|
"__Note__: This method returns the secret value. Take care not to write the the secret value to output."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"retrieved_secret = keyvault.get_secret(name=\"secret-name\")\n",
|
|
"local_secret==retrieved_secret"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In submitted runs on local and remote compute, you can use the get_secret method of Run instance to get the secret value from Key Vault. \n",
|
|
"\n",
|
|
"The method gives you a simple shortcut: the Run instance is aware of its Workspace and Keyvault, so it can directly obtain the secret without you having to instantiate the Workspace and Keyvault within remote run.\n",
|
|
"\n",
|
|
"__Note__: This method returns the secret value. Take care not to write the secret to output.\n",
|
|
"\n",
|
|
"For example, let's create a simple script _get_secret.py_ that gets the secret we set earlier. In an actual appication, you would use the secret, for example to access a database or other password-protected resource."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%writefile get_secret.py\n",
|
|
"\n",
|
|
"from azureml.core import Run\n",
|
|
"\n",
|
|
"run = Run.get_context()\n",
|
|
"secret_value = run.get_secret(name=\"secret-name\")\n",
|
|
"print(\"Got secret value {} , but don't write it out!\".format(len(secret_value) * \"*\"))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Then, submit the script as a regular script run, and find the obfuscated secret value in run output. You can use the same approach to other kinds of runs, such as Estimator ones."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core import Experiment\n",
|
|
"from azureml.core.script_run_config import ScriptRunConfig\n",
|
|
"\n",
|
|
"exp = Experiment(workspace = ws, name=\"try-secret\")\n",
|
|
"src = ScriptRunConfig(source_directory=\".\", script=\"get_secret.py\")\n",
|
|
"\n",
|
|
"run = exp.submit(src)\n",
|
|
"run.wait_for_completion(show_output=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Furthermore, you can set and get multiple secrets using set_secrets and get_secrets methods."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"authors": [
|
|
{
|
|
"name": "roastala"
|
|
}
|
|
],
|
|
"kernelspec": {
|
|
"display_name": "Python 3.6",
|
|
"language": "python",
|
|
"name": "python36"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.9"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
} |