Updated notebook folders
This commit is contained in:
Roope Astala
@@ -1,625 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
||||
"\n",
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Model Development with Custom Weights"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This example shows how to retrain a model with custom weights and fine-tune the model with quantization, then deploy the model running on FPGA. Only Windows is supported. We use TensorFlow and Keras to build our model. We are going to use transfer learning, with ResNet50 as a featurizer. We don't use the last layer of ResNet50 in this case and instead add our own classification layer using Keras.\n",
|
||||
"\n",
|
||||
"The custom wegiths are trained with ImageNet on ResNet50. We will use the Kaggle Cats and Dogs dataset to retrain and fine-tune the model. The dataset can be downloaded [here](https://www.microsoft.com/en-us/download/details.aspx?id=54765). Download the zip and extract to a directory named 'catsanddogs' under your user directory (\"~/catsanddogs\"). \n",
|
||||
"\n",
|
||||
"Please set up your environment as described in the [quick start](project-brainwave-quickstart.ipynb)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import sys\n",
|
||||
"import tensorflow as tf\n",
|
||||
"import numpy as np\n",
|
||||
"from keras import backend as K"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup Environment\n",
|
||||
"After you train your model in float32, you'll write the weights to a place on disk. We also need a location to store the models that get downloaded."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"custom_weights_dir = os.path.expanduser(\"~/custom-weights\")\n",
|
||||
"saved_model_dir = os.path.expanduser(\"~/models\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Prepare Data\n",
|
||||
"Load the files we are going to use for training and testing. By default this notebook uses only a very small subset of the Cats and Dogs dataset. That makes it run relatively quickly."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import glob\n",
|
||||
"import imghdr\n",
|
||||
"datadir = os.path.expanduser(\"~/catsanddogs\")\n",
|
||||
"\n",
|
||||
"cat_files = glob.glob(os.path.join(datadir, 'PetImages', 'Cat', '*.jpg'))\n",
|
||||
"dog_files = glob.glob(os.path.join(datadir, 'PetImages', 'Dog', '*.jpg'))\n",
|
||||
"\n",
|
||||
"# Limit the data set to make the notebook execute quickly.\n",
|
||||
"cat_files = cat_files[:64]\n",
|
||||
"dog_files = dog_files[:64]\n",
|
||||
"\n",
|
||||
"# The data set has a few images that are not jpeg. Remove them.\n",
|
||||
"cat_files = [f for f in cat_files if imghdr.what(f) == 'jpeg']\n",
|
||||
"dog_files = [f for f in dog_files if imghdr.what(f) == 'jpeg']\n",
|
||||
"\n",
|
||||
"if(not len(cat_files) or not len(dog_files)):\n",
|
||||
" print(\"Please download the Kaggle Cats and Dogs dataset form https://www.microsoft.com/en-us/download/details.aspx?id=54765 and extract the zip to \" + datadir) \n",
|
||||
" raise ValueError(\"Data not found\")\n",
|
||||
"else:\n",
|
||||
" print(cat_files[0])\n",
|
||||
" print(dog_files[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Construct a numpy array as labels\n",
|
||||
"image_paths = cat_files + dog_files\n",
|
||||
"total_files = len(cat_files) + len(dog_files)\n",
|
||||
"labels = np.zeros(total_files)\n",
|
||||
"labels[len(cat_files):] = 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Split images data as training data and test data\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"onehot_labels = np.array([[0,1] if i else [1,0] for i in labels])\n",
|
||||
"img_train, img_test, label_train, label_test = train_test_split(image_paths, onehot_labels, random_state=42, shuffle=True)\n",
|
||||
"\n",
|
||||
"print(len(img_train), len(img_test), label_train.shape, label_test.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Construct Model\n",
|
||||
"We use ResNet50 for the featuirzer and build our own classifier using Keras layers. We train the featurizer and the classifier as one model. The weights trained on ImageNet are used as the starting point for the retraining of our featurizer. The weights are loaded from tensorflow chkeckpoint files."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Before passing image dataset to the ResNet50 featurizer, we need to preprocess the input file to get it into the form expected by ResNet50. ResNet50 expects float tensors representing the images in BGR, channel last order. We've provided a default implementation of the preprocessing that you can use."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import azureml.contrib.brainwave.models.utils as utils\n",
|
||||
"\n",
|
||||
"def preprocess_images():\n",
|
||||
" # Convert images to 3D tensors [width,height,channel] - channels are in BGR order.\n",
|
||||
" in_images = tf.placeholder(tf.string)\n",
|
||||
" image_tensors = utils.preprocess_array(in_images)\n",
|
||||
" return in_images, image_tensors"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We use Keras layer APIs to construct the classifier. Because we're using the tensorflow backend, we can train this classifier in one session with our Resnet50 model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_classifier(in_tensor):\n",
|
||||
" from keras.layers import Dropout, Dense, Flatten\n",
|
||||
" K.set_session(tf.get_default_session())\n",
|
||||
" \n",
|
||||
" FC_SIZE = 1024\n",
|
||||
" NUM_CLASSES = 2\n",
|
||||
"\n",
|
||||
" x = Dropout(0.2, input_shape=(1, 1, 2048,))(in_tensor)\n",
|
||||
" x = Dense(FC_SIZE, activation='relu', input_dim=(1, 1, 2048,))(x)\n",
|
||||
" x = Flatten()(x)\n",
|
||||
" preds = Dense(NUM_CLASSES, activation='softmax', input_dim=FC_SIZE, name='classifier_output')(x)\n",
|
||||
" return preds"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now every component of the model is defined, we can construct the model. Constructing the model with the project brainwave models is two steps - first we import the graph definition, then we restore the weights of the model into a tensorflow session. Because the quantized graph defintion and the float32 graph defintion share the same node names in the graph definitions, we can initally train the weights in float32, and then reload them with the quantized operations (which take longer) to fine-tune the model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def construct_model(quantized, starting_weights_directory = None):\n",
|
||||
" from azureml.contrib.brainwave.models import Resnet50, QuantizedResnet50\n",
|
||||
" \n",
|
||||
" # Convert images to 3D tensors [width,height,channel]\n",
|
||||
" in_images, image_tensors = preprocess_images()\n",
|
||||
"\n",
|
||||
" # Construct featurizer using quantized or unquantized ResNet50 model\n",
|
||||
" if not quantized:\n",
|
||||
" featurizer = Resnet50(saved_model_dir)\n",
|
||||
" else:\n",
|
||||
" featurizer = QuantizedResnet50(saved_model_dir, custom_weights_directory = starting_weights_directory)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" features = featurizer.import_graph_def(input_tensor=image_tensors)\n",
|
||||
" # Construct classifier\n",
|
||||
" preds = construct_classifier(features)\n",
|
||||
" \n",
|
||||
" # Initialize weights\n",
|
||||
" sess = tf.get_default_session()\n",
|
||||
" tf.global_variables_initializer().run()\n",
|
||||
"\n",
|
||||
" featurizer.restore_weights(sess)\n",
|
||||
"\n",
|
||||
" return in_images, image_tensors, features, preds, featurizer"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Train Model\n",
|
||||
"First we train the model with custom weights but without quantization. Training is done with native float precision (32-bit floats). We load the traing data set and batch the training with 10 epochs. When the performance reaches desired level or starts decredation, we stop the training iteration and save the weights as tensorflow checkpoint files. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def read_files(files):\n",
|
||||
" \"\"\" Read files to array\"\"\"\n",
|
||||
" contents = []\n",
|
||||
" for path in files:\n",
|
||||
" with open(path, 'rb') as f:\n",
|
||||
" contents.append(f.read())\n",
|
||||
" return contents"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def train_model(preds, in_images, img_train, label_train, is_retrain = False, train_epoch = 10):\n",
|
||||
" \"\"\" training model \"\"\"\n",
|
||||
" from keras.objectives import binary_crossentropy\n",
|
||||
" from tqdm import tqdm\n",
|
||||
" \n",
|
||||
" learning_rate = 0.001 if is_retrain else 0.01\n",
|
||||
" \n",
|
||||
" # Specify the loss function\n",
|
||||
" in_labels = tf.placeholder(tf.float32, shape=(None, 2)) \n",
|
||||
" cross_entropy = tf.reduce_mean(binary_crossentropy(in_labels, preds))\n",
|
||||
" optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cross_entropy)\n",
|
||||
"\n",
|
||||
" def chunks(a, b, n):\n",
|
||||
" \"\"\"Yield successive n-sized chunks from a and b.\"\"\"\n",
|
||||
" if (len(a) != len(b)):\n",
|
||||
" print(\"a and b are not equal in chunks(a,b,n)\")\n",
|
||||
" raise ValueError(\"Parameter error\")\n",
|
||||
"\n",
|
||||
" for i in range(0, len(a), n):\n",
|
||||
" yield a[i:i + n], b[i:i + n]\n",
|
||||
"\n",
|
||||
" chunk_size = 16\n",
|
||||
" chunk_num = len(label_train) / chunk_size\n",
|
||||
"\n",
|
||||
" sess = tf.get_default_session()\n",
|
||||
" for epoch in range(train_epoch):\n",
|
||||
" avg_loss = 0\n",
|
||||
" for img_chunk, label_chunk in tqdm(chunks(img_train, label_train, chunk_size)):\n",
|
||||
" contents = read_files(img_chunk)\n",
|
||||
" _, loss = sess.run([optimizer, cross_entropy],\n",
|
||||
" feed_dict={in_images: contents,\n",
|
||||
" in_labels: label_chunk,\n",
|
||||
" K.learning_phase(): 1})\n",
|
||||
" avg_loss += loss / chunk_num\n",
|
||||
" print(\"Epoch:\", (epoch + 1), \"loss = \", \"{:.3f}\".format(avg_loss))\n",
|
||||
" \n",
|
||||
" # Reach desired performance\n",
|
||||
" if (avg_loss < 0.001):\n",
|
||||
" break"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def test_model(preds, in_images, img_test, label_test):\n",
|
||||
" \"\"\"Test the model\"\"\"\n",
|
||||
" from keras.metrics import categorical_accuracy\n",
|
||||
"\n",
|
||||
" in_labels = tf.placeholder(tf.float32, shape=(None, 2))\n",
|
||||
" accuracy = tf.reduce_mean(categorical_accuracy(in_labels, preds))\n",
|
||||
" contents = read_files(img_test)\n",
|
||||
"\n",
|
||||
" accuracy = accuracy.eval(feed_dict={in_images: contents,\n",
|
||||
" in_labels: label_test,\n",
|
||||
" K.learning_phase(): 0})\n",
|
||||
" return accuracy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Launch the training\n",
|
||||
"tf.reset_default_graph()\n",
|
||||
"sess = tf.Session(graph=tf.get_default_graph())\n",
|
||||
"\n",
|
||||
"with sess.as_default():\n",
|
||||
" in_images, image_tensors, features, preds, featurizer = construct_model(quantized=False)\n",
|
||||
" train_model(preds, in_images, img_train, label_train, is_retrain=False, train_epoch=10) \n",
|
||||
" accuracy = test_model(preds, in_images, img_test, label_test) \n",
|
||||
" print(\"Accuracy:\", accuracy)\n",
|
||||
" featurizer.save_weights(custom_weights_dir + \"/rn50\", tf.get_default_session())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Test Model\n",
|
||||
"After training, we evaluate the trained model's accuracy on test dataset with quantization. So that we know the model's performance if it is deployed on the FPGA."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"tf.reset_default_graph()\n",
|
||||
"sess = tf.Session(graph=tf.get_default_graph())\n",
|
||||
"\n",
|
||||
"with sess.as_default():\n",
|
||||
" print(\"Testing trained model with quantization\")\n",
|
||||
" in_images, image_tensors, features, preds, quantized_featurizer = construct_model(quantized=True, starting_weights_directory=custom_weights_dir)\n",
|
||||
" accuracy = test_model(preds, in_images, img_test, label_test) \n",
|
||||
" print(\"Accuracy:\", accuracy)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Fine-Tune Model\n",
|
||||
"Sometimes, the model's accuracy can drop significantly after quantization. In those cases, we need to retrain the model enabled with quantization to get better model accuracy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"if (accuracy < 0.93):\n",
|
||||
" with sess.as_default():\n",
|
||||
" print(\"Fine-tuning model with quantization\")\n",
|
||||
" train_model(preds, in_images, img_train, label_train, is_retrain=True, train_epoch=10)\n",
|
||||
" accuracy = test_model(preds, in_images, img_test, label_test) \n",
|
||||
" print(\"Accuracy:\", accuracy)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Service Definition\n",
|
||||
"Like in the QuickStart notebook our service definition pipeline consists of three stages. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.pipeline import ModelDefinition, TensorflowStage, BrainWaveStage\n",
|
||||
"\n",
|
||||
"model_def_path = os.path.join(saved_model_dir, 'model_def.zip')\n",
|
||||
"\n",
|
||||
"model_def = ModelDefinition()\n",
|
||||
"model_def.pipeline.append(TensorflowStage(sess, in_images, image_tensors))\n",
|
||||
"model_def.pipeline.append(BrainWaveStage(sess, quantized_featurizer))\n",
|
||||
"model_def.pipeline.append(TensorflowStage(sess, features, preds))\n",
|
||||
"model_def.save(model_def_path)\n",
|
||||
"print(model_def_path)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Deploy\n",
|
||||
"Go to our [GitHub repo](https://aka.ms/aml-real-time-ai) \"docs\" folder to learn how to create a Model Management Account and find the required information below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"ws = Workspace.from_config()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The first time the code below runs it will create a new service running your model. If you want to change the model you can make changes above in this notebook and save a new service definition. Then this code will update the running service in place to run the new model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.model import Model\n",
|
||||
"from azureml.core.image import Image\n",
|
||||
"from azureml.core.webservice import Webservice\n",
|
||||
"from azureml.contrib.brainwave import BrainwaveWebservice, BrainwaveImage\n",
|
||||
"from azureml.exceptions import WebserviceException\n",
|
||||
"\n",
|
||||
"model_name = \"catsanddogs-resnet50-model\"\n",
|
||||
"image_name = \"catsanddogs-resnet50-image\"\n",
|
||||
"service_name = \"modelbuild-service\"\n",
|
||||
"\n",
|
||||
"registered_model = Model.register(ws, model_def_path, model_name)\n",
|
||||
"\n",
|
||||
"image_config = BrainwaveImage.image_configuration()\n",
|
||||
"deployment_config = BrainwaveWebservice.deploy_configuration()\n",
|
||||
" \n",
|
||||
"try:\n",
|
||||
" service = Webservice(ws, service_name)\n",
|
||||
" service.delete()\n",
|
||||
" service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)\n",
|
||||
" service.wait_for_deployment(True)\n",
|
||||
"except WebserviceException:\n",
|
||||
" service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)\n",
|
||||
" service.wait_for_deployment(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The service is now running in Azure and ready to serve requests. We can check the address and port."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(service.ipAddress + ':' + str(service.port))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Client\n",
|
||||
"There is a simple test client at amlrealtimeai.PredictionClient which can be used for testing. We'll use this client to score an image with our new service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.client import PredictionClient\n",
|
||||
"client = PredictionClient(service.ipAddress, service.port)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can adapt the client [code](../../pythonlib/amlrealtimeai/client.py) to meet your needs. There is also an example C# [client](../../sample-clients/csharp).\n",
|
||||
"\n",
|
||||
"The service provides an API that is compatible with TensorFlow Serving. There are instructions to download a sample client [here](https://www.tensorflow.org/serving/setup)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Request\n",
|
||||
"Let's see how our service does on a few images. It may get a few wrong."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Specify an image to classify\n",
|
||||
"print('CATS')\n",
|
||||
"for image_file in cat_files[:8]:\n",
|
||||
" results = client.score_image(image_file)\n",
|
||||
" result = 'CORRECT ' if results[0] > results[1] else 'WRONG '\n",
|
||||
" print(result + str(results))\n",
|
||||
"print('DOGS')\n",
|
||||
"for image_file in dog_files[:8]:\n",
|
||||
" results = client.score_image(image_file)\n",
|
||||
" result = 'CORRECT ' if results[1] > results[0] else 'WRONG '\n",
|
||||
" print(result + str(results))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Cleanup\n",
|
||||
"Run the cell below to delete your service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"service.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Appendix"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"License for plot_confusion_matrix:\n",
|
||||
"\n",
|
||||
"New BSD License\n",
|
||||
"\n",
|
||||
"Copyright (c) 2007-2018 The scikit-learn developers.\n",
|
||||
"All rights reserved.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Redistribution and use in source and binary forms, with or without\n",
|
||||
"modification, are permitted provided that the following conditions are met:\n",
|
||||
"\n",
|
||||
" a. Redistributions of source code must retain the above copyright notice,\n",
|
||||
" this list of conditions and the following disclaimer.\n",
|
||||
" b. Redistributions in binary form must reproduce the above copyright\n",
|
||||
" notice, this list of conditions and the following disclaimer in the\n",
|
||||
" documentation and/or other materials provided with the distribution.\n",
|
||||
" c. Neither the name of the Scikit-learn Developers nor the names of\n",
|
||||
" its contributors may be used to endorse or promote products\n",
|
||||
" derived from this software without specific prior written\n",
|
||||
" permission. \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\n",
|
||||
"AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n",
|
||||
"IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n",
|
||||
"ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR\n",
|
||||
"ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n",
|
||||
"DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n",
|
||||
"SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n",
|
||||
"CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\n",
|
||||
"LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n",
|
||||
"OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n",
|
||||
"DAMAGE.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"authors": [
|
||||
{
|
||||
"name": "coverste"
|
||||
}
|
||||
],
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.6",
|
||||
"language": "python",
|
||||
"name": "python36"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.5.2"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,312 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
||||
"\n",
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Azure ML Hardware Accelerated Models Quickstart"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This tutorial will show you how to deploy an image recognition service based on the ResNet 50 classifier in just a few minutes using the Azure Machine Learning Accelerated AI service. Get more help from our [documentation](https://aka.ms/aml-real-time-ai) or [forum](https://aka.ms/aml-forum).\n",
|
||||
"\n",
|
||||
"We will use an accelerated ResNet50 featurizer running on an FPGA. This functionality is powered by Project Brainwave, which handles translating deep neural networks (DNN) into an FPGA program.\n",
|
||||
"\n",
|
||||
"## Request Quota\n",
|
||||
"**IMPORTANT:** You must [request quota](https://aka.ms/aml-real-time-ai-request) and be approved before you can successfully run this notebook. Notebook 00 will show you how to create a workspace which you can use to request quota."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Imports"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import tensorflow as tf"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Image preprocessing\n",
|
||||
"We'd like our service to accept JPEG images as input. However the input to ResNet50 is a tensor. So we need code that decodes JPEG images and does the preprocessing required by ResNet50. The Accelerated AI service can execute TensorFlow graphs as part of the service and we'll use that ability to do the image preprocessing. This code defines a TensorFlow graph that preprocesses an array of JPEG images (as strings) and produces a tensor that is ready to be featurized by ResNet50."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Input images as a two-dimensional tensor containing an arbitrary number of images represented a strings\n",
|
||||
"import azureml.contrib.brainwave.models.utils as utils\n",
|
||||
"in_images = tf.placeholder(tf.string)\n",
|
||||
"image_tensors = utils.preprocess_array(in_images)\n",
|
||||
"print(image_tensors.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Featurizer\n",
|
||||
"We use ResNet50 as a featurizer. In this step we initialize the model. This downloads a TensorFlow checkpoint of the quantized ResNet50."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.models import QuantizedResnet50\n",
|
||||
"model_path = os.path.expanduser('~/models')\n",
|
||||
"model = QuantizedResnet50(model_path, is_frozen = True)\n",
|
||||
"feature_tensor = model.import_graph_def(image_tensors)\n",
|
||||
"print(model.version)\n",
|
||||
"print(feature_tensor.name)\n",
|
||||
"print(feature_tensor.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Classifier\n",
|
||||
"The model we downloaded includes a classifier which takes the output of the ResNet50 and identifies an image. This classifier is trained on the ImageNet dataset. We are going to use this classifier for our service. The next [notebook](project-brainwave-trainsfer-learning.ipynb) shows how to train a classifier for a different data set. The input to the classifier is a tensor matching the output of our ResNet50 featurizer."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"classifier_output = model.get_default_classifier(feature_tensor)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Service Definition\n",
|
||||
"Now that we've definied the image preprocessing, featurizer, and classifier that we will execute on our service we can create a service definition. The service definition is a set of files generated from the model that allow us to deploy to the FPGA service. The service definition consists of a pipeline. The pipeline is a series of stages that are executed in order. We support TensorFlow stages, Keras stages, and BrainWave stages. The stages will be executed in order on the service, with the output of each stage input into the subsequent stage.\n",
|
||||
"\n",
|
||||
"To create a TensorFlow stage we specify a session containing the graph (in this case we are using the default graph) and the input and output tensors to this stage. We use this information to save the graph so that we can execute it on the service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.pipeline import ModelDefinition, TensorflowStage, BrainWaveStage\n",
|
||||
"\n",
|
||||
"save_path = os.path.expanduser('~/models/save')\n",
|
||||
"model_def_path = os.path.join(save_path, 'model_def.zip')\n",
|
||||
"\n",
|
||||
"model_def = ModelDefinition()\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" model_def.pipeline.append(TensorflowStage(sess, in_images, image_tensors))\n",
|
||||
" model_def.pipeline.append(BrainWaveStage(sess, model))\n",
|
||||
" model_def.pipeline.append(TensorflowStage(sess, feature_tensor, classifier_output))\n",
|
||||
" model_def.save(model_def_path)\n",
|
||||
" print(model_def_path)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Deploy\n",
|
||||
"Time to create a service from the service definition. You need a Workspace in the **East US 2** location. In the previous notebooks, you've created this Workspace. The code below will load that Workspace from a configuration file."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"ws = Workspace.from_config()\n",
|
||||
"print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Upload the model to the workspace."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.model import Model\n",
|
||||
"model_name = \"resnet-50-rtai\"\n",
|
||||
"registered_model = Model.register(ws, model_def_path, model_name)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Create a service from the model that we registered. If this is a new service then we create it. If you already have a service with this name then the existing service will be updated to use this model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.webservice import Webservice\n",
|
||||
"from azureml.exceptions import WebserviceException\n",
|
||||
"from azureml.contrib.brainwave import BrainwaveWebservice, BrainwaveImage\n",
|
||||
"service_name = \"imagenet-infer\"\n",
|
||||
"service = None\n",
|
||||
"try:\n",
|
||||
" service = Webservice(ws, service_name)\n",
|
||||
"except WebserviceException:\n",
|
||||
" image_config = BrainwaveImage.image_configuration()\n",
|
||||
" deployment_config = BrainwaveWebservice.deploy_configuration()\n",
|
||||
" service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)\n",
|
||||
" service.wait_for_deployment(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Client\n",
|
||||
"The service supports gRPC and the TensorFlow Serving \"predict\" API. We provide a client that can call the service to get predictions on aka.ms/rtai. You can also invoke the service like any other web service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To understand the results we need a mapping to the human readable imagenet classes"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"classes_entries = requests.get(\"https://raw.githubusercontent.com/Lasagne/Recipes/master/examples/resnet50/imagenet_classes.txt\").text.splitlines()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We can now send an image to the service and get the predictions. Let's see if it can identify a snow leopard.\n",
|
||||
"\n",
|
||||
"Snow leopard in a zoo. Photo by Peter Bolliger.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"results = service.run('snowleopardgaze.jpg')\n",
|
||||
"# map results [class_id] => [confidence]\n",
|
||||
"results = enumerate(results)\n",
|
||||
"# sort results by confidence\n",
|
||||
"sorted_results = sorted(results, key=lambda x: x[1], reverse=True)\n",
|
||||
"# print top 5 results\n",
|
||||
"for top in sorted_results[:5]:\n",
|
||||
" print(classes_entries[top[0]], 'confidence:', top[1])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Cleanup\n",
|
||||
"Run the cell below to delete your service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"service.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Congratulations! You've just created a service that does predictions using an FPGA. The next [notebook](project-brainwave-trainsfer-learning.ipynb) shows how to customize the service using transfer learning to classify different types of images."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"authors": [
|
||||
{
|
||||
"name": "coverste"
|
||||
}
|
||||
],
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.6",
|
||||
"language": "python",
|
||||
"name": "python36"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.5.2"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -1,572 +0,0 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Copyright (c) Microsoft Corporation. All rights reserved.\n",
|
||||
"\n",
|
||||
"Licensed under the MIT License."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Model Development"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This example shows how to build, train, evaluate and deploy a model running on FPGA. Only Windows is supported. We use TensorFlow and Keras to build our model. We are going to use transfer learning, with ResNet152 as a featurizer. We don't use the last layer of ResNet152 in this case and instead add and train our own classification layer.\n",
|
||||
"\n",
|
||||
"We will use the Kaggle Cats and Dogs dataset to train the classifier. The dataset can be downloaded [here](https://www.microsoft.com/en-us/download/details.aspx?id=54765). Download the zip and extract to a directory named 'catsanddogs' under your user directory (\"~/catsanddogs\").\n",
|
||||
"\n",
|
||||
"Please set up your environment as described in the [quick start](project-brainwave-quickstart.ipynb)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import tensorflow as tf\n",
|
||||
"import numpy as np"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Model Construction\n",
|
||||
"Load the files we are going to use for training and testing. By default this notebook uses only a very small subset of the Cats and Dogs dataset. That makes it run quickly, but doesn't create a very accurate classifier. You can improve the classifier by using more of the dataset."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import glob\n",
|
||||
"import imghdr\n",
|
||||
"datadir = os.path.expanduser(\"~/catsanddogs\")\n",
|
||||
"\n",
|
||||
"cat_files = glob.glob(os.path.join(datadir, 'PetImages', 'Cat', '*.jpg'))\n",
|
||||
"dog_files = glob.glob(os.path.join(datadir, 'PetImages', 'Dog', '*.jpg'))\n",
|
||||
"\n",
|
||||
"# Limit the data set to make the notebook execute quickly.\n",
|
||||
"cat_files = cat_files[:64]\n",
|
||||
"dog_files = dog_files[:64]\n",
|
||||
"\n",
|
||||
"# The data set has a few images that are not jpeg. Remove them.\n",
|
||||
"cat_files = [f for f in cat_files if imghdr.what(f) == 'jpeg']\n",
|
||||
"dog_files = [f for f in dog_files if imghdr.what(f) == 'jpeg']\n",
|
||||
"\n",
|
||||
"if(not len(cat_files) or not len(dog_files)):\n",
|
||||
" print(\"Please download the Kaggle Cats and Dogs dataset form https://www.microsoft.com/en-us/download/details.aspx?id=54765 and extract the zip to \" + datadir) \n",
|
||||
" raise ValueError(\"Data not found\")\n",
|
||||
"else:\n",
|
||||
" print(cat_files[0])\n",
|
||||
" print(dog_files[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# constructing a numpy array as labels\n",
|
||||
"image_paths = cat_files + dog_files\n",
|
||||
"total_files = len(cat_files) + len(dog_files)\n",
|
||||
"labels = np.zeros(total_files)\n",
|
||||
"labels[len(cat_files):] = 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We need to preprocess the input file to get it into the form expected by ResNet152. We've provided a default implementation of the preprocessing that you can use."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Input images as a two-dimensional tensor containing an arbitrary number of images represented a strings\n",
|
||||
"import azureml.contrib.brainwave.models.utils as utils\n",
|
||||
"in_images = tf.placeholder(tf.string)\n",
|
||||
"image_tensors = utils.preprocess_array(in_images)\n",
|
||||
"print(image_tensors.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Alternatively, if you would like to customize the preprocessing, you can write your own preprocessor using TensorFlow operations.\n",
|
||||
"\n",
|
||||
"The input to the classifier we are training is the set of features produced by ResNet50. To train the classifier we need to \n",
|
||||
"featurize the images using ResNet50. You can also run the featurizer locally on CPU or GPU. We import the featurizer as frozen, so that we are only training the classifier."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.models import QuantizedResnet152\n",
|
||||
"model_path = os.path.expanduser('~/models')\n",
|
||||
"bwmodel = QuantizedResnet152(model_path, is_frozen = True)\n",
|
||||
"print(bwmodel.version)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Calling import_graph_def on the featurizer will create a service that runs the featurizer on FPGA."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"features = bwmodel.import_graph_def(input_tensor=image_tensors)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Pre-compute features\n",
|
||||
"Load the data set and compute the features. These can be precomputed because they don't change during training. This can take a while to run on CPU."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"def chunks(l, n):\n",
|
||||
" \"\"\"Yield successive n-sized chunks from l.\"\"\"\n",
|
||||
" for i in range(0, len(l), n):\n",
|
||||
" yield l[i:i + n]\n",
|
||||
"\n",
|
||||
"def read_files(files):\n",
|
||||
" contents = []\n",
|
||||
" for path in files:\n",
|
||||
" with open(path, 'rb') as f:\n",
|
||||
" contents.append(f.read())\n",
|
||||
" return contents\n",
|
||||
" \n",
|
||||
"feature_list = []\n",
|
||||
"with tf.Session() as sess:\n",
|
||||
" for chunk in tqdm(chunks(image_paths, 5)):\n",
|
||||
" contents = read_files(chunk)\n",
|
||||
" result = sess.run([features], feed_dict={in_images: contents})\n",
|
||||
" feature_list.extend(result[0])\n",
|
||||
"\n",
|
||||
"feature_results = np.array(feature_list)\n",
|
||||
"print(feature_results.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Add and Train the classifier\n",
|
||||
"We use Keras to define and train a simple classifier."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from keras.models import Sequential\n",
|
||||
"from keras.layers import Dropout, Dense, Flatten\n",
|
||||
"from keras import optimizers\n",
|
||||
"\n",
|
||||
"FC_SIZE = 1024\n",
|
||||
"NUM_CLASSES = 2\n",
|
||||
"\n",
|
||||
"model = Sequential()\n",
|
||||
"model.add(Dropout(0.2, input_shape=(1, 1, 2048,)))\n",
|
||||
"model.add(Dense(FC_SIZE, activation='relu', input_dim=(1, 1, 2048,)))\n",
|
||||
"model.add(Flatten())\n",
|
||||
"model.add(Dense(NUM_CLASSES, activation='sigmoid', input_dim=FC_SIZE))\n",
|
||||
"\n",
|
||||
"model.compile(optimizer=optimizers.SGD(lr=1e-4,momentum=0.9), loss='binary_crossentropy', metrics=['accuracy'])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Prepare the train and test data."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"onehot_labels = np.array([[0,1] if i else [1,0] for i in labels])\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(feature_results, onehot_labels, random_state=42, shuffle=True)\n",
|
||||
"print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Train the classifier."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model.fit(X_train, y_train, epochs=16, batch_size=32)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Test the Classifier\n",
|
||||
"Let's test the classifier and see how well it does. Since we only trained on a few images, we are not expecting to win a Kaggle competition, but it will likely get most of the images correct. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from numpy import argmax\n",
|
||||
"\n",
|
||||
"y_probs = model.predict(X_test)\n",
|
||||
"y_prob_max = np.argmax(y_probs, 1)\n",
|
||||
"y_test_max = np.argmax(y_test, 1)\n",
|
||||
"print(y_prob_max)\n",
|
||||
"print(y_test_max)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from sklearn.metrics import confusion_matrix, roc_auc_score, accuracy_score, precision_score, recall_score, f1_score\n",
|
||||
"import itertools\n",
|
||||
"import matplotlib\n",
|
||||
"from matplotlib import pyplot as plt\n",
|
||||
"\n",
|
||||
"# compute a bunch of classification metrics \n",
|
||||
"def classification_metrics(y_true, y_pred, y_prob):\n",
|
||||
" cm_dict = {}\n",
|
||||
" cm_dict['Accuracy'] = accuracy_score(y_true, y_pred)\n",
|
||||
" cm_dict['Precision'] = precision_score(y_true, y_pred)\n",
|
||||
" cm_dict['Recall'] = recall_score(y_true, y_pred)\n",
|
||||
" cm_dict['F1'] = f1_score(y_true, y_pred) \n",
|
||||
" cm_dict['AUC'] = roc_auc_score(y_true, y_prob[:,0])\n",
|
||||
" cm_dict['Confusion Matrix'] = confusion_matrix(y_true, y_pred).tolist()\n",
|
||||
" return cm_dict\n",
|
||||
"\n",
|
||||
"def plot_confusion_matrix(cm, classes, normalize=False, title='Confusion matrix', cmap=plt.cm.Blues):\n",
|
||||
" \"\"\"Plots a confusion matrix.\n",
|
||||
" Source: http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html\n",
|
||||
" New BSD License - see appendix\n",
|
||||
" \"\"\"\n",
|
||||
" cm_max = cm.max()\n",
|
||||
" cm_min = cm.min()\n",
|
||||
" if cm_min > 0: cm_min = 0\n",
|
||||
" if normalize:\n",
|
||||
" cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
|
||||
" cm_max = 1\n",
|
||||
" plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
|
||||
" plt.title(title)\n",
|
||||
" plt.colorbar()\n",
|
||||
" tick_marks = np.arange(len(classes))\n",
|
||||
" plt.xticks(tick_marks, classes, rotation=45)\n",
|
||||
" plt.yticks(tick_marks, classes)\n",
|
||||
" thresh = cm_max / 2.\n",
|
||||
" plt.clim(cm_min, cm_max)\n",
|
||||
"\n",
|
||||
" for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
|
||||
" plt.text(j, i,\n",
|
||||
" round(cm[i, j], 3), # round to 3 decimals if they are float\n",
|
||||
" horizontalalignment=\"center\",\n",
|
||||
" color=\"white\" if cm[i, j] > thresh else \"black\")\n",
|
||||
" plt.ylabel('True label')\n",
|
||||
" plt.xlabel('Predicted label')\n",
|
||||
" plt.show()\n",
|
||||
" \n",
|
||||
"cm_dict = classification_metrics(y_test_max, y_prob_max, y_probs)\n",
|
||||
"for m in cm_dict:\n",
|
||||
" print(m, cm_dict[m])\n",
|
||||
"cm = np.asarray(cm_dict['Confusion Matrix'])\n",
|
||||
"plot_confusion_matrix(cm, ['fail','pass'], normalize=False)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Service Definition\n",
|
||||
"Like in the QuickStart notebook our service definition pipeline consists of three stages. Because the preprocessing and featurizing stage don't contain any variables, we can use a default session.\n",
|
||||
"Here we use the Keras classifier as the final stage."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.pipeline import ModelDefinition, TensorflowStage, BrainWaveStage, KerasStage\n",
|
||||
"\n",
|
||||
"model_def = ModelDefinition()\n",
|
||||
"model_def.pipeline.append(TensorflowStage(tf.Session(), in_images, image_tensors))\n",
|
||||
"model_def.pipeline.append(BrainWaveStage(tf.Session(), bwmodel))\n",
|
||||
"model_def.pipeline.append(KerasStage(model))\n",
|
||||
"\n",
|
||||
"model_def_path = os.path.join(datadir, 'save', 'model_def')\n",
|
||||
"model_def.save(model_def_path)\n",
|
||||
"print(model_def_path)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Deploy"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.model import Model\n",
|
||||
"from azureml.core import Workspace\n",
|
||||
"\n",
|
||||
"ws = Workspace.from_config()\n",
|
||||
"print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')\n",
|
||||
"model_name = \"catsanddogs-model\"\n",
|
||||
"service_name = \"modelbuild-service\"\n",
|
||||
"\n",
|
||||
"registered_model = Model.register(ws, model_def_path, model_name)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The first time the code below runs it will create a new service running your model. If you want to change the model you can make changes above in this notebook and save a new service definition. Then this code will update the running service in place to run the new model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.core.webservice import Webservice\n",
|
||||
"from azureml.exceptions import WebserviceException\n",
|
||||
"from azureml.contrib.brainwave import BrainwaveWebservice, BrainwaveImage\n",
|
||||
"try:\n",
|
||||
" service = Webservice(ws, service_name)\n",
|
||||
"except WebserviceException:\n",
|
||||
" image_config = BrainwaveImage.image_configuration()\n",
|
||||
" deployment_config = BrainwaveWebservice.deploy_configuration()\n",
|
||||
" service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)\n",
|
||||
" service.wait_for_deployment(True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The service is now running in Azure and ready to serve requests. We can check the address and port."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(service.ipAddress + ':' + str(service.port))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Client\n",
|
||||
"There is a simple test client at amlrealtimeai.PredictionClient which can be used for testing. We'll use this client to score an image with our new service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from azureml.contrib.brainwave.client import PredictionClient\n",
|
||||
"client = PredictionClient(service.ipAddress, service.port)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can adapt the client [code](../../pythonlib/amlrealtimeai/client.py) to meet your needs. There is also an example C# [client](../../sample-clients/csharp).\n",
|
||||
"\n",
|
||||
"The service provides an API that is compatible with TensorFlow Serving. There are instructions to download a sample client [here](https://www.tensorflow.org/serving/setup)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Request\n",
|
||||
"Let's see how our service does on a few images. It may get a few wrong."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Specify an image to classify\n",
|
||||
"print('CATS')\n",
|
||||
"for image_file in cat_files[:8]:\n",
|
||||
" results = client.score_image(image_file)\n",
|
||||
" result = 'CORRECT ' if results[0] > results[1] else 'WRONG '\n",
|
||||
" print(result + str(results))\n",
|
||||
"print('DOGS')\n",
|
||||
"for image_file in dog_files[:8]:\n",
|
||||
" results = client.score_image(image_file)\n",
|
||||
" result = 'CORRECT ' if results[1] > results[0] else 'WRONG '\n",
|
||||
" print(result + str(results))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Cleanup\n",
|
||||
"Run the cell below to delete your service. In the [next notebook](project-brainwave-custom-weights.ipynb) you will learn how to retrain all the weights of one of the models"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"service.delete()\n",
|
||||
" \n",
|
||||
"registered_model.delete()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Appendix"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"License for plot_confusion_matrix:\n",
|
||||
"\n",
|
||||
"New BSD License\n",
|
||||
"\n",
|
||||
"Copyright (c) 2007\u00e2\u20ac\u201c2018 The scikit-learn developers.\n",
|
||||
"All rights reserved.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Redistribution and use in source and binary forms, with or without\n",
|
||||
"modification, are permitted provided that the following conditions are met:\n",
|
||||
"\n",
|
||||
" a. Redistributions of source code must retain the above copyright notice,\n",
|
||||
" this list of conditions and the following disclaimer.\n",
|
||||
" b. Redistributions in binary form must reproduce the above copyright\n",
|
||||
" notice, this list of conditions and the following disclaimer in the\n",
|
||||
" documentation and/or other materials provided with the distribution.\n",
|
||||
" c. Neither the name of the Scikit-learn Developers nor the names of\n",
|
||||
" its contributors may be used to endorse or promote products\n",
|
||||
" derived from this software without specific prior written\n",
|
||||
" permission. \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\n",
|
||||
"AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n",
|
||||
"IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE\n",
|
||||
"ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR\n",
|
||||
"ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n",
|
||||
"DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n",
|
||||
"SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n",
|
||||
"CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT\n",
|
||||
"LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY\n",
|
||||
"OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\n",
|
||||
"DAMAGE.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"authors": [
|
||||
{
|
||||
"name": "coverste"
|
||||
}
|
||||
],
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.6",
|
||||
"language": "python",
|
||||
"name": "python36"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.5.2"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Reference in New Issue
Block a user