jprdonnelly/MachineLearningNotebooks
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update notebooks

Browse Source
This commit is contained in:
Roope Astala
2018-09-21 16:03:50 -04:00
parent 48f037d6dd
commit 87bb03f7a3
23 changed files with 2724 additions and 247 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
tutorials/01.train-models.ipynb
View File

@@ -98,7 +98,7 @@
"source": [
"### Create experiment\n",
"\n",
"Create an experiment to track the runs in your workspace. A workspace can have muliple experiments; an experiment must belongn to a workspace."
"Create an experiment to track the runs in your workspace. A workspace can have muliple experiments. "
]
},
{
@@ -121,9 +121,7 @@
"\n",
"Azure Azure ML Managed Compute is a managed service that enables data scientists to train machine learning models on clusters of Azure virtual machines, including VMs with GPU support. In this tutorial, you create an Azure Managed Compute cluster as your training environment. This code creates a cluster for you if it does not already exist in your workspace. \n",
"\n",
" **Creation of the cluster takes approximately 5 minutes.** If the cluster is already in the workspace this code uses it and skips the creation process.\n",
"\n",
"**Note**: As with other Azure services, there are limits on certain resources (for eg. BatchAI cluster size) associated with the Azure Machine Learning service. Please read [this article](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-manage-quotas) on the default limits and how to request more quota."
" **Creation of the cluster takes approximately 5 minutes.** If the cluster is already in the workspace this code uses it and skips the creation process."
]
},
{
@@ -146,7 +144,7 @@
"try:\n",
" # look for the existing cluster by name\n",
" compute_target = ComputeTarget(workspace=ws, name=batchai_cluster_name)\n",
" if compute_target is BatchAiCompute:\n",
" if type(compute_target) is BatchAiCompute:\n",
" print('found compute target {}, just use it.'.format(batchai_cluster_name))\n",
" else:\n",
" print('{} exists but it is not a Batch AI cluster. Please choose a different name.'.format(batchai_cluster_name))\n",
@@ -188,6 +186,13 @@
"Download the MNIST dataset and save the files into a `data` directory locally. Images and labels for both training and testing are downloaded."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
@@ -330,7 +335,7 @@
"\n",
"### Create a directory\n",
"\n",
"Create a directory to hold all script files are other assets."
"Create a directory to deliver the necessary code from your computer to the remote resource."
]
},
{
@@ -434,7 +439,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Copy the utility library that loads the dataset into the script folder to be accessed by the training script."
"The file `utils.py` is referenced from the training script to load the dataset correctly. Copy this script into the script folder so that it can be accessed along with the training script on the remote resource."
]
},
{
@@ -457,11 +462,12 @@
"\n",
"* The name of the estimator object, `est`\n",
"* The directory that contains your scripts. All the files in this directory are uploaded into the cluster nodes for execution. \n",
"* The compute target. In this case you will use the Managed Compute cluster you created\n",
"* The compute target. In this case you will use the Batch AI cluster you created\n",
"* The training script name, train.py\n",
"* The `data-folder` parameter used by the training script to access the data\n",
"* Any Python packages needed for training\n",
"In this tutorial, this target is the Managed Compute cluster. All files in the script folder are uploaded into the cluster nodes for execution."
"* Parameters required from the training script \n",
"* Python packages needed for training\n",
"\n",
"In this tutorial, this target is the Batch AI cluster. All files in the script folder are uploaded into the cluster nodes for execution. The data_folder is set to use the datastore (`ds.as_mount()`)."
]
},
{
@@ -507,7 +513,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Since the call is asynchronous, it returns a **Preparing** or **running** state as soon as the job is started.\n",
"Since the call is asynchronous, it returns a **Preparing** or **Running** state as soon as the job is started.\n",
"\n",
"## Monitor a remote run\n",
"\n",
@@ -595,7 +601,7 @@
"\n",
"## Register model\n",
"\n",
"The last step in the training script wrote the file `outputs/sklearn_mnist_model.pkl` in a folder named `outputs` in the VM of the cluster where the job is executed. `outputs` is a special folder in that all content in the `outputs` directory is automatically uploaded as part of the run record in the experiment under your workspace. Hence, the model file is now also available in your workspace. \n",
"The last step in the training script wrote the file `outputs/sklearn_mnist_model.pkl` in a directory named `outputs` in the VM of the cluster where the job is executed. `outputs` is a special directory in that all content in this directory is automatically uploaded to your workspace. This content appears in the run record in the experiment under your workspace. Hence, the model file is now also available in your workspace.\n",
"\n",
"You can see files associated with that run."
]