{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "Copyright (c) Microsoft Corporation. All rights reserved.\n", "\n", "Licensed under the MIT License." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/deploy-to-local/register-model-deploy-local-advanced.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Register model and deploy locally with advanced usages\n", "\n", "This example shows how to deploy a web service in step-by-step fashion:\n", "\n", " 1. Register model\n", " 2. Deploy the image as a web service in a local Docker container.\n", " 3. Quickly test changes to your entry script by reloading the local service.\n", " 4. Optionally, you can also make changes to model, conda or extra_docker_file_steps and update local service" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisites\n", "If you are using an Azure Machine Learning Notebook VM, you are all set. Otherwise, make sure you go through the [configuration](../../../configuration.ipynb) Notebook first if you haven't." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Check core SDK version number\n", "import azureml.core\n", "\n", "print(\"SDK version:\", azureml.core.VERSION)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialize Workspace\n", "\n", "Initialize a workspace object from persisted configuration." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "create workspace" ] }, "outputs": [], "source": [ "from azureml.core import Workspace\n", "\n", "ws = Workspace.from_config()\n", "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Register Model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can add tags and descriptions to your models. we are using `sklearn_regression_model.pkl` file in the current directory as a model with the same name `sklearn_regression_model.pkl` in the workspace.\n", "\n", "Using tags, you can track useful information such as the name and version of the machine learning library used to train the model, framework, category, target customer etc. Note that tags must be alphanumeric." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "register model from file" ] }, "outputs": [], "source": [ "from azureml.core.model import Model\n", "\n", "model = Model.register(model_path = \"sklearn_regression_model.pkl\",\n", " model_name = \"sklearn_regression_model.pkl\",\n", " tags = {'area': \"diabetes\", 'type': \"regression\"},\n", " description = \"Ridge regression model to predict diabetes\",\n", " workspace = ws)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Manage your dependencies in a folder" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "source_directory = \"C:/abc\"\n", "\n", "os.makedirs(source_directory, exist_ok = True)\n", "os.makedirs(\"C:/abc/x/y\", exist_ok = True)\n", "os.makedirs(\"C:/abc/env\", exist_ok = True)\n", "os.makedirs(\"C:/abc/dockerstep\", exist_ok = True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Show `score.py`. Note that the `sklearn_regression_model.pkl` in the `get_model_path` call is referring to a model named `sklearn_regression_model.pkl` registered under the workspace. It is NOT referencing the local file." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile C:/abc/x/y/score.py\n", "import pickle\n", "import json\n", "import numpy as np\n", "from sklearn.externals import joblib\n", "from sklearn.linear_model import Ridge\n", "from azureml.core.model import Model\n", "\n", "from inference_schema.schema_decorators import input_schema, output_schema\n", "from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType\n", "\n", "def init():\n", " global model\n", " # note here \"sklearn_regression_model.pkl\" is the name of the model registered under\n", " # this is a different behavior than before when the code is run locally, even though the code is the same.\n", " model_path = Model.get_model_path('sklearn_regression_model.pkl')\n", " # deserialize the model file back into a sklearn model\n", " model = joblib.load(model_path)\n", " global name\n", " # note here, entire source directory on inference config gets added into image\n", " # bellow is the example how you can use any extra files in image\n", " with open('./abc/extradata.json') as json_file: \n", " data = json.load(json_file)\n", " name = data[\"people\"][0][\"name\"]\n", "\n", "input_sample = np.array([[10,9,8,7,6,5,4,3,2,1]])\n", "output_sample = np.array([3726.995])\n", "\n", "@input_schema('data', NumpyParameterType(input_sample))\n", "@output_schema(NumpyParameterType(output_sample))\n", "def run(data):\n", " try:\n", " result = model.predict(data)\n", " # you can return any datatype as long as it is JSON-serializable\n", " return \"Hello \" + name + \" here is your result = \" + str(result)\n", " except Exception as e:\n", " error = str(e)\n", " return error" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile C:/abc/env/myenv.yml\n", "name: project_environment\n", "dependencies:\n", " - python=3.6.2\n", " - pip:\n", " - azureml-defaults\n", " - scikit-learn\n", " - numpy\n", " - inference-schema[numpy-support]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile C:/abc/dockerstep/customDockerStep.txt\n", "RUN echo \"this is test\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile C:/abc/extradata.json\n", "{\n", " \"people\": [\n", " {\n", " \"website\": \"microsoft.com\", \n", " \"from\": \"Seattle\", \n", " \"name\": \"Mrudula\"\n", " }\n", " ]\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create Inference Configuration\n", "\n", " - source_directory = holds source path as string, this entire folder gets added in image so its really easy to access any files within this folder or subfolder\n", " - runtime = Which runtime to use for the image. Current supported runtimes are 'spark-py' and 'python\n", " - entry_script = contains logic specific to initializing your model and running predictions\n", " - conda_file = manages conda and python package dependencies.\n", " - extra_docker_file_steps = optional: any extra steps you want to inject into docker file" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from azureml.core.model import InferenceConfig\n", "\n", "inference_config = InferenceConfig(source_directory=\"C:/abc\",\n", " runtime= \"python\", \n", " entry_script=\"x/y/score.py\",\n", " conda_file=\"env/myenv.yml\", \n", " extra_docker_file_steps=\"dockerstep/customDockerStep.txt\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Deploy Model as a Local Docker Web Service\n", "\n", "*Make sure you have Docker installed and running.*\n", "\n", "Note that the service creation can take few minutes.\n", "\n", "NOTE:\n", "\n", "we require docker running with linux container. If you are running Docker for Windows, you need to ensure the Linux Engine is running\n", "\n", " powershell command to switch to linux engine\n", " & 'C:\\Program Files\\Docker\\Docker\\DockerCli.exe' -SwitchLinuxEngine\n", "\n", "and c drive is shared https://docs.docker.com/docker-for-windows/#shared-drives\n", "sometimes you have to reshare c drive as docker \n", "\n", "" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "deploy service", "aci" ] }, "outputs": [], "source": [ "from azureml.core.webservice import LocalWebservice\n", "\n", "#this is optional, if not provided we choose random port\n", "deployment_config = LocalWebservice.deploy_configuration(port=6789)\n", "\n", "local_service = Model.deploy(ws, \"test\", [model], inference_config, deployment_config)\n", "\n", "local_service.wait_for_deployment()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print('Local service port: {}'.format(local_service.port))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Check Status and Get Container Logs\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(local_service.get_logs())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Web Service" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Call the web service with some input data to get a prediction." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import json\n", "\n", "sample_input = json.dumps({\n", " 'data': [\n", " [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n", " [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]\n", " ]\n", "})\n", "\n", "sample_input = bytes(sample_input, encoding='utf-8')\n", "\n", "print(local_service.run(input_data=sample_input))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reload Service\n", "\n", "You can update your score.py file and then call `reload()` to quickly restart the service. This will only reload your execution script and dependency files, it will not rebuild the underlying Docker image. As a result, `reload()` is fast, but if you do need to rebuild the image -- to add a new Conda or pip package, for instance -- you will have to call `update()`, instead (see below)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%writefile C:/abc/x/y/score.py\n", "import pickle\n", "import json\n", "import numpy as np\n", "from sklearn.externals import joblib\n", "from sklearn.linear_model import Ridge\n", "from azureml.core.model import Model\n", "\n", "from inference_schema.schema_decorators import input_schema, output_schema\n", "from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType\n", "\n", "def init():\n", " global model\n", " # note here \"sklearn_regression_model.pkl\" is the name of the model registered under\n", " # this is a different behavior than before when the code is run locally, even though the code is the same.\n", " model_path = Model.get_model_path('sklearn_regression_model.pkl')\n", " # deserialize the model file back into a sklearn model\n", " model = joblib.load(model_path)\n", " global name, from_location\n", " # note here, entire source directory on inference config gets added into image\n", " # bellow is the example how you can use any extra files in image\n", " with open('./abc/extradata.json') as json_file: \n", " data = json.load(json_file)\n", " name = data[\"people\"][0][\"name\"]\n", " from_location = data[\"people\"][0][\"from\"]\n", "\n", "input_sample = np.array([[10,9,8,7,6,5,4,3,2,1]])\n", "output_sample = np.array([3726.995])\n", "\n", "@input_schema('data', NumpyParameterType(input_sample))\n", "@output_schema(NumpyParameterType(output_sample))\n", "def run(data):\n", " try:\n", " result = model.predict(data)\n", " # you can return any datatype as long as it is JSON-serializable\n", " return \"Hello \" + name + \" from \" + from_location + \" here is your result = \" + str(result)\n", " except Exception as e:\n", " error = str(e)\n", " return error" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "local_service.reload()\n", "print(\"--------------------------------------------------------------\")\n", "\n", "# after reload now if you call run this will return updated return message\n", "\n", "print(local_service.run(input_data=sample_input))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Update Service\n", "\n", "If you want to change your model(s), Conda dependencies, or deployment configuration, call `update()` to rebuild the Docker image.\n", "\n", "```python\n", "\n", "local_service.update(models = [SomeOtherModelObject],\n", " deployment_config = local_config,\n", " inference_config = inference_config)\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Delete Service" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "local_service.delete()" ] } ], "metadata": { "authors": [ { "name": "raymondl" } ], "kernelspec": { "display_name": "Python 3.6", "language": "python", "name": "python36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" } }, "nbformat": 4, "nbformat_minor": 2 }