mirror of
https://github.com/Azure/MachineLearningNotebooks.git
synced 2025-12-20 01:27:06 -05:00
494 lines
21 KiB
Plaintext
494 lines
21 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
|
"\n",
|
|
"Licensed under the MIT License."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Azure ML Hardware Accelerated Object Detection"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"This tutorial will show you how to deploy an object detection service based on the SSD-VGG model in just a few minutes using the Azure Machine Learning Accelerated AI service.\n",
|
|
"\n",
|
|
"We will use the SSD-VGG model accelerated on an FPGA. Our Accelerated Models Service handles translating deep neural networks (DNN) into an FPGA program.\n",
|
|
"\n",
|
|
"The steps in this notebook are: \n",
|
|
"1. [Setup Environment](#set-up-environment)\n",
|
|
"* [Construct Model](#construct-model)\n",
|
|
" * Image Preprocessing\n",
|
|
" * Featurizer\n",
|
|
" * Save Model\n",
|
|
" * Save input and output tensor names\n",
|
|
"* [Create Image](#create-image)\n",
|
|
"* [Deploy Image](#deploy-image)\n",
|
|
"* [Test the Service](#test-service)\n",
|
|
" * Create Client\n",
|
|
" * Serve the model\n",
|
|
"* [Cleanup](#cleanup)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"set-up-environment\"></a>\n",
|
|
"## 1. Set up Environment\n",
|
|
"### 1.a. Imports"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import os\n",
|
|
"import tensorflow as tf"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 1.b. Retrieve Workspace\n",
|
|
"If you haven't created a Workspace, please follow [this notebook](\"../../../configuration.ipynb\") to do so. If you have, run the codeblock below to retrieve it. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core import Workspace\n",
|
|
"\n",
|
|
"ws = Workspace.from_config()\n",
|
|
"print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"construct-model\"></a>\n",
|
|
"## 2. Construct model\n",
|
|
"### 2.a. Image preprocessing\n",
|
|
"We'd like our service to accept JPEG images as input. However the input to SSD-VGG is a float tensor of shape \\[1, 300, 300, 3\\]. The first dimension is batch, then height, width, and channels (i.e. NHWC). To bridge this gap, we need code that decodes JPEG images and resizes them appropriately for input to SSD-VGG. The Accelerated AI service can execute TensorFlow graphs as part of the service and we'll use that ability to do the image preprocessing. This code defines a TensorFlow graph that preprocesses an array of JPEG images (as TensorFlow strings) and produces a tensor that is ready to be featurized by SSD-VGG.\n",
|
|
"\n",
|
|
"**Note:** Expect to see TF deprecation warnings until we port our SDK over to use Tensorflow 2.0."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Input images as a two-dimensional tensor containing an arbitrary number of images represented a strings\n",
|
|
"import azureml.accel.models.utils as utils\n",
|
|
"tf.reset_default_graph()\n",
|
|
"\n",
|
|
"in_images = tf.placeholder(tf.string)\n",
|
|
"image_tensors = utils.preprocess_array(in_images, output_width=300, output_height=300, preserve_aspect_ratio=False)\n",
|
|
"print(image_tensors.shape)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2.b. Featurizer\n",
|
|
"The SSD-VGG model is different from our other models in that it generates 12 tensor outputs. These corresponds to x,y displacements of the anchor boxes and the detection confidence (for 21 classes). Because these outputs are not convenient to work with, we will later use a pre-defined post-processing utility to transform the outputs into a simplified list of bounding boxes with their respective class and confidence.\n",
|
|
"\n",
|
|
"For more information about the output tensors, take this example: the output tensor 'ssd_300_vgg/block4_box/Reshape_1:0' has a shape of [None, 37, 37, 4, 21]. This gives the pre-softmax confidence for 4 anchor boxes situated at each site of a 37 x 37 grid imposed on the image, one confidence score for each of the 21 classes. The first dimension is the batch dimension. Likewise, 'ssd_300_vgg/block4_box/Reshape:0' has shape [None, 37, 37, 4, 4] and encodes the (cx, cy) center shift and rescaling (sw, sh) relative to each anchor box. Refer to the [SSD-VGG paper](https://arxiv.org/abs/1512.02325) to understand how these are computed. The other 10 tensors are defined similarly."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.accel.models import SsdVgg\n",
|
|
"\n",
|
|
"saved_model_dir = os.path.join(os.path.expanduser('~'), 'models')\n",
|
|
"model_graph = SsdVgg(saved_model_dir, is_frozen = True)\n",
|
|
"\n",
|
|
"print('SSD-VGG Input Tensors:')\n",
|
|
"for idx, input_name in enumerate(model_graph.input_tensor_list):\n",
|
|
" print('{}, {}'.format(input_name, model_graph.get_input_dims(idx)))\n",
|
|
" \n",
|
|
"print('SSD-VGG Output Tensors:')\n",
|
|
"for idx, output_name in enumerate(model_graph.output_tensor_list):\n",
|
|
" print('{}, {}'.format(output_name, model_graph.get_output_dims(idx)))\n",
|
|
"\n",
|
|
"ssd_outputs = model_graph.import_graph_def(image_tensors, is_training=False)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2.c. Save Model\n",
|
|
"Now that we loaded both parts of the tensorflow graph (preprocessor and SSD-VGG featurizer), we can save the graph and associated variables to a directory which we can register as an Azure ML Model."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"model_name = \"ssdvgg\"\n",
|
|
"model_save_path = os.path.join(saved_model_dir, model_name, \"saved_model\")\n",
|
|
"print(\"Saving model in {}\".format(model_save_path))\n",
|
|
"\n",
|
|
"output_map = {}\n",
|
|
"for i, output in enumerate(ssd_outputs):\n",
|
|
" output_map['out_{}'.format(i)] = output\n",
|
|
"\n",
|
|
"with tf.Session() as sess:\n",
|
|
" model_graph.restore_weights(sess)\n",
|
|
" tf.saved_model.simple_save(sess, \n",
|
|
" model_save_path, \n",
|
|
" inputs={'images': in_images}, \n",
|
|
" outputs=output_map)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2.d. Important! Save names of input and output tensors\n",
|
|
"\n",
|
|
"These input and output tensors that were created during the preprocessing and classifier steps are also going to be used when **converting the model** to an Accelerated Model that can run on FPGA's and for **making an inferencing request**. It is very important to save this information!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"register model from file"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"input_tensors = in_images.name\n",
|
|
"# We will use the list of output tensors during inferencing\n",
|
|
"output_tensors = [output.name for output in ssd_outputs]\n",
|
|
"# However, for multiple output tensors, our AccelOnnxConverter will \n",
|
|
"# accept comma-delimited strings (lists will cause error)\n",
|
|
"output_tensors_str = \",\".join(output_tensors)\n",
|
|
"\n",
|
|
"print(input_tensors)\n",
|
|
"print(output_tensors)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"create-image\"></a>\n",
|
|
"## 3. Create AccelContainerImage\n",
|
|
"Below we will execute all the same steps as in the [Quickstart](./accelerated-models-quickstart.ipynb#create-image) to package the model we have saved locally into an accelerated Docker image saved in our workspace. To complete all the steps, it may take a few minutes. For more details on each step, check out the [Quickstart section on model registration](./accelerated-models-quickstart.ipynb#register-model)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core import Workspace\n",
|
|
"from azureml.core.model import Model\n",
|
|
"from azureml.core.image import Image\n",
|
|
"from azureml.accel import AccelOnnxConverter\n",
|
|
"from azureml.accel import AccelContainerImage\n",
|
|
"\n",
|
|
"# Retrieve workspace\n",
|
|
"ws = Workspace.from_config()\n",
|
|
"print(\"Successfully retrieved workspace:\", ws.name, ws.resource_group, ws.location, ws.subscription_id, '\\n')\n",
|
|
"\n",
|
|
"# Register model\n",
|
|
"registered_model = Model.register(workspace = ws,\n",
|
|
" model_path = model_save_path,\n",
|
|
" model_name = model_name)\n",
|
|
"print(\"Successfully registered: \", registered_model.name, registered_model.description, registered_model.version, '\\n', sep = '\\t')\n",
|
|
"\n",
|
|
"# Convert model\n",
|
|
"convert_request = AccelOnnxConverter.convert_tf_model(ws, registered_model, input_tensors, output_tensors_str)\n",
|
|
"# If it fails, you can run wait_for_completion again with show_output=True.\n",
|
|
"convert_request.wait_for_completion(show_output=False)\n",
|
|
"converted_model = convert_request.result\n",
|
|
"print(\"\\nSuccessfully converted: \", converted_model.name, converted_model.url, converted_model.version, \n",
|
|
" converted_model.id, converted_model.created_time, '\\n')\n",
|
|
"\n",
|
|
"# Package into AccelContainerImage\n",
|
|
"image_config = AccelContainerImage.image_configuration()\n",
|
|
"# Image name must be lowercase\n",
|
|
"image_name = \"{}-image\".format(model_name)\n",
|
|
"image = Image.create(name = image_name,\n",
|
|
" models = [converted_model],\n",
|
|
" image_config = image_config, \n",
|
|
" workspace = ws)\n",
|
|
"image.wait_for_creation()\n",
|
|
"print(\"Created AccelContainerImage: {} {} {}\\n\".format(image.name, image.creation_state, image.image_location))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"deploy-image\"></a>\n",
|
|
"## 4. Deploy image\n",
|
|
"Once you have an Azure ML Accelerated Image in your Workspace, you can deploy it to two destinations, to a Databox Edge machine or to an AKS cluster. \n",
|
|
"\n",
|
|
"### 4.a. Deploy to Databox Edge Machine using IoT Hub\n",
|
|
"See the sample [here](https://github.com/Azure-Samples/aml-real-time-ai/) for using the Azure IoT CLI extension for deploying your Docker image to your Databox Edge Machine.\n",
|
|
"\n",
|
|
"### 4.b. Deploy to AKS Cluster\n",
|
|
"Same as in the [Quickstart section on image deployment](./accelerated-models-quickstart.ipynb#deploy-image), we are going to create an AKS cluster with FPGA-enabled machines, then deploy our service to it.\n",
|
|
"#### Create AKS ComputeTarget"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core.compute import AksCompute, ComputeTarget\n",
|
|
"\n",
|
|
"# Uses the specific FPGA enabled VM (sku: Standard_PB6s)\n",
|
|
"# Standard_PB6s are available in: eastus, westus2, westeurope, southeastasia\n",
|
|
"prov_config = AksCompute.provisioning_configuration(vm_size = \"Standard_PB6s\",\n",
|
|
" agent_count = 1, \n",
|
|
" location = \"eastus\")\n",
|
|
"\n",
|
|
"aks_name = 'aks-pb6-obj'\n",
|
|
"# Create the cluster\n",
|
|
"aks_target = ComputeTarget.create(workspace = ws, \n",
|
|
" name = aks_name, \n",
|
|
" provisioning_configuration = prov_config)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Provisioning an AKS cluster might take awhile (15 or so minutes), and we want to wait until it's successfully provisioned before we can deploy a service to it. If you interrupt this cell, provisioning of the cluster will continue. You can re-run it or check the status in your Workspace under Compute."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"aks_target.wait_for_completion(show_output = True)\n",
|
|
"print(aks_target.provisioning_state)\n",
|
|
"print(aks_target.provisioning_errors)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Deploy AccelContainerImage to AKS ComputeTarget"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from azureml.core.webservice import Webservice, AksWebservice\n",
|
|
"\n",
|
|
"# Set the web service configuration (for creating a test service, we don't want autoscale enabled)\n",
|
|
"# Authentication is enabled by default, but for testing we specify False\n",
|
|
"aks_config = AksWebservice.deploy_configuration(autoscale_enabled=False,\n",
|
|
" num_replicas=1,\n",
|
|
" auth_enabled = False)\n",
|
|
"\n",
|
|
"aks_service_name ='my-aks-service'\n",
|
|
"\n",
|
|
"aks_service = Webservice.deploy_from_image(workspace = ws,\n",
|
|
" name = aks_service_name,\n",
|
|
" image = image,\n",
|
|
" deployment_config = aks_config,\n",
|
|
" deployment_target = aks_target)\n",
|
|
"aks_service.wait_for_deployment(show_output = True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"test-service\"></a>\n",
|
|
"## 5. Test the service\n",
|
|
"<a id=\"create-client\"></a>\n",
|
|
"### 5.a. Create Client\n",
|
|
"The image supports gRPC and the TensorFlow Serving \"predict\" API. We have a client that can call into the docker image to get predictions. \n",
|
|
"\n",
|
|
"**Note:** If you chose to use auth_enabled=True when creating your AksWebservice.deploy_configuration(), see documentation [here](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice(class)?view=azure-ml-py#get-keys--) on how to retrieve your keys and use either key as an argument to PredictionClient(...,access_token=key).",
|
|
"\n",
|
|
"**WARNING:** If you are running on Azure Notebooks free compute, you will not be able to make outgoing calls to your service. Try locating your client on a different machine to consume it."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Using the grpc client in AzureML Accelerated Models SDK\n",
|
|
"from azureml.accel.client import PredictionClient\n",
|
|
"\n",
|
|
"address = aks_service.scoring_uri\n",
|
|
"ssl_enabled = address.startswith(\"https\")\n",
|
|
"address = address[address.find('/')+2:].strip('/')\n",
|
|
"port = 443 if ssl_enabled else 80\n",
|
|
"\n",
|
|
"# Initialize AzureML Accelerated Models client\n",
|
|
"client = PredictionClient(address=address,\n",
|
|
" port=port,\n",
|
|
" use_ssl=ssl_enabled,\n",
|
|
" service_name=aks_service.name)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can adapt the client [code](https://github.com/Azure/aml-real-time-ai/blob/master/pythonlib/amlrealtimeai/client.py) to meet your needs. There is also an example C# [client](https://github.com/Azure/aml-real-time-ai/blob/master/sample-clients/csharp).\n",
|
|
"\n",
|
|
"The service provides an API that is compatible with TensorFlow Serving. There are instructions to download a sample client [here](https://www.tensorflow.org/serving/setup)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"serve-model\"></a>\n",
|
|
"### 5.b. Serve the model\n",
|
|
"The SSD-VGG model returns the confidence and bounding boxes for all possible anchor boxes. As mentioned earlier, we will use a post-processing routine to transform this into a list of bounding boxes (y1, x1, y2, x2) where x, y are fractional coordinates measured from left and top respectively. A respective list of classes and scores is also returned to tag each bounding box. Below we make use of this information to draw the bounding boxes on top the original image. Note that in the post-processing routine we select a confidence threshold of 0.5."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import cv2\n",
|
|
"from matplotlib import pyplot as plt\n",
|
|
"\n",
|
|
"colors_tableau = [(255, 255, 255), (31, 119, 180), (174, 199, 232), (255, 127, 14), (255, 187, 120),\n",
|
|
" (44, 160, 44), (152, 223, 138), (214, 39, 40), (255, 152, 150),\n",
|
|
" (148, 103, 189), (197, 176, 213), (140, 86, 75), (196, 156, 148),\n",
|
|
" (227, 119, 194), (247, 182, 210), (127, 127, 127), (199, 199, 199),\n",
|
|
" (188, 189, 34), (219, 219, 141), (23, 190, 207), (158, 218, 229)]\n",
|
|
"\n",
|
|
"\n",
|
|
"def draw_boxes_on_img(img, classes, scores, bboxes, thickness=2):\n",
|
|
" shape = img.shape\n",
|
|
" for i in range(bboxes.shape[0]):\n",
|
|
" bbox = bboxes[i]\n",
|
|
" color = colors_tableau[classes[i]]\n",
|
|
" # Draw bounding box...\n",
|
|
" p1 = (int(bbox[0] * shape[0]), int(bbox[1] * shape[1]))\n",
|
|
" p2 = (int(bbox[2] * shape[0]), int(bbox[3] * shape[1]))\n",
|
|
" cv2.rectangle(img, p1[::-1], p2[::-1], color, thickness)\n",
|
|
" # Draw text...\n",
|
|
" s = '%s/%.3f' % (classes[i], scores[i])\n",
|
|
" p1 = (p1[0]-5, p1[1])\n",
|
|
" cv2.putText(img, s, p1[::-1], cv2.FONT_HERSHEY_DUPLEX, 0.4, color, 1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import azureml.accel._external.ssdvgg_utils as ssdvgg_utils\n",
|
|
"\n",
|
|
"result = client.score_file(path=\"meeting.jpg\", input_name=input_tensors, outputs=output_tensors)\n",
|
|
"classes, scores, bboxes = ssdvgg_utils.postprocess(result, select_threshold=0.5)\n",
|
|
"\n",
|
|
"img = cv2.imread('meeting.jpg', 1)\n",
|
|
"img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n",
|
|
"draw_boxes_on_img(img, classes, scores, bboxes)\n",
|
|
"plt.imshow(img)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<a id=\"cleanup\"></a>\n",
|
|
"## 6. Cleanup\n",
|
|
"It's important to clean up your resources, so that you won't incur unnecessary costs. In the [next notebook](./accelerated-models-training.ipynb) you will learn how to train a classfier on a new dataset using transfer learning."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"aks_service.delete()\n",
|
|
"aks_target.delete()\n",
|
|
"image.delete()\n",
|
|
"registered_model.delete()\n",
|
|
"converted_model.delete()"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"authors": [
|
|
{
|
|
"name": "coverste"
|
|
},
|
|
{
|
|
"name": "paledger"
|
|
},
|
|
{
|
|
"name": "sukha"
|
|
}
|
|
],
|
|
"kernelspec": {
|
|
"display_name": "Python 3.6",
|
|
"language": "python",
|
|
"name": "python36"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.0"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
} |